A hybrid fuzzy regression model and its application in hydrology engineering

The goal of this paper is to handle the large variation issues in fuzzy data by constructing a variable spread multivariate adaptive regression splines (MARS) fuzzy regression model with crisp parameters estimation and fuzzy error terms. It deals with imprecise measurement of response variable and crisp measurement of explanatory variables. The proposed method is a two-phase procedure which applies the MARS technique at phase one and an optimization problem at phase two to estimate the center and fuzziness of the response variable. The proposed method, therefore, handles two problems simultaneously: the problem of large variation issue and the problem of variation spreads in fuzzy observations. A realistic application of the proposed method is also presented, by which the suspended load is modeled using discharge in a hydrology engineering problem. Empirical results demonstrate that the proposed approach is more efficient and more realistic than some well-known least-squares fuzzy regression models.     

Fuzzy Regression Models Using the Least-Squares Method Based on the Concept of Distance

Fuzzy regression models are developed to construct the relationship between explanatory variables and responses in a fuzzy environment. In order to increase the explanatory performance of the model, the least-squares method is applied to determine the numeric coefficients based on the concept of distance. Unlike most existing approaches, the numeric coefficients in the proposed model can have negative values. The proposed model minimizes total estimation error in terms of the sum of the average squared distance between the observed and estimated responses based on a few $alpha$-cuts. The proposed approach is not limited to triangular fuzzy numbers; it can be used to carry out a large number of fuzzy observations efficiently because the model is based on traditional statistical methods. Comparisons with existing methods show that based on the total estimation error using the mean squared error and Kim and Bishu's criterion, the explanatory performance of the proposed model is satisfactory.      

A class of fuzzy clusterwise regression models

In this paper we introduce a class of fuzzy clusterwise regression models with LR fuzzy response variable and numeric explanatory variables, which embodies fuzzy clustering, into a fuzzy regression framework. The model bypasses the heterogeneity problem that could arise in fuzzy regression by subdividing the dataset into homogeneous clusters and performing separate fuzzy regression on each cluster. The integration of the clustering model into the regression framework allows us to simultaneously estimate the regression parameters and the membership degree of each observation to each cluster by optimizing a single objective function. The class of models proposed here includes, as special cases, the fuzzy clusterwise linear regression model and the fuzzy clusterwise polynomial regression model. We also introduce a set of goodness of fit indices to evaluate the fit of the regression model within each cluster as well as in the whole dataset. Finally, we consider some cluster validity criteria that are useful in identifying the “optimal” number of clusters. Several applications are provided in order to illustrate the approach.     

Fuzzy logistic regression based on the least squares approach with application in clinical studies

To model fuzzy binary observations, a new model named “Fuzzy Logistic Regression” is proposed and discussed in this study. In fact, due to the vague nature of binary observations, no probability distribution can be considered for these data. Therefore, the ordinary logistic regression may not be appropriate. This study attempts to construct a fuzzy model based on possibility of success. These possibilities are defined by some linguistic terms such as …, low, medium, high…. Then, by use of the Extension principle, the logarithm transformation of “possibilistic odds” is modeled based on a set of crisp explanatory variables observations. Also, to estimate parameters in the proposed model, the least squares method in fuzzy linear regression is used. For evaluating the model, a criterion named the “capability index” is calculated. At the end, because of widespread applications of logistic regression in clinical studies and also, the abundance of vague observations in clinical diagnosis, the suspected cases to Systematic Lupus Erythematosus (SLE) disease is modeled based on some significant risk factors to detect the application of the model. The results showed that the proposed model could be a rational substituted model of an ordinary one in modeling the clinical vague status.     

A mathematical programming method for formulating a fuzzy regression model based on distance criterion.

Fuzzy regression models are useful to investigate the relationship between explanatory and response variables with fuzzy observations. Different from previous studies, this correspondence proposes a mathematical programming method to construct a fuzzy regression model based on a distance criterion. The objective of the mathematical programming is to minimize the sum of distances between the estimated and observed responses on the X axis, such that the fuzzy regression model constructed has the minimal total estimation error in distance. Only several alpha-cuts of fuzzy observations are needed as inputs to the mathematical programming model; therefore, the applications are not restricted to triangular fuzzy numbers. Three examples, adopted in the previous studies, and a larger example, modified from the crisp case, are used to illustrate the performance of the proposed approach. The results indicate that the proposed model has better performance than those in the previous studies based on either distance criterion or Kim and Bishu's criterion. In addition, the efficiency and effectiveness for solving the larger example by the proposed model are also satisfactory.     

ARMA(p,q) type high order fuzzy time series forecast method based on fuzzy logic relations

Within classic time series approaches, a time series model can be studied under 3 groups, namely AR (autoregressive model), MA (moving averages model) and ARMA (autoregressive moving averages model). On the other hand, solutions are based mostly on fuzzy AR time series models in the fuzzy time series literature. However, just a few fuzzy ARMA time series models have proposed until now. Fuzzy AR time series models have been divided into two groups named first order and high order models in the literature, highlighting the impact of model degree on forecast performance. However, model structure has been disregarded in these fuzzy AR models. Therefore, it is necessary to eliminate the model specification error arising from not utilizing of MA variables in the fuzzy time series approaches. For this reason, a new high order fuzzy ARMA(p,q) time series solution algorithm based on fuzzy logic group relations including fuzzy MA variables along with fuzzy AR variables has been proposed in this study. The main purpose of this article is to show that the forecast performance can be significantly improved when the deficiency of not utilizing MA variables. The other aim is also to show that the proposed method is better than the other fuzzy ARMA time series models in the literature from the point of forecast performance. Therefore, the new proposed method has been compared regarding forecast performance against some methods commonly used in literature by applying them on gold prices in Turkey, Istanbul Stock Exchange (IMKB) and the Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX).     

Error measures for fuzzy linear regression: Monte Carlo simulation approach

The focus of this study is to use Monte Carlo method in fuzzy linear regression. The purpose of the study is to figure out the appropriate error measures for the estimation of fuzzy linear regression model parameters with Monte Carlo method. Since model parameters are estimated without any mathematical programming or heavy fuzzy arithmetic operations in fuzzy linear regression with Monte Carlo method. In the literature, only two error measures (E₁ and E₂) are available for the estimation of fuzzy linear regression model parameters. Additionally, accuracy of available error measures under the Monte Carlo procedure has not been evaluated. In this article, mean square error, mean percentage error, mean absolute percentage error, and symmetric mean absolute percentage error are proposed for the estimation of fuzzy linear regression model parameters with Monte Carlo method. Moreover, estimation accuracies of existing and proposed error measures are explored. Error measures are compared to each other in terms of estimation accuracy; hence, this study demonstrates that the best error measures to estimate fuzzy linear regression model parameters with Monte Carlo method are proved to be E₁, E₂, and the mean square error. One the other hand, the worst one can be given as the mean percentage error. These results would be useful to enrich the studies that have already focused on fuzzy linear regression models.     

A variable spread fuzzy linear regression model with higher explanatory power and forecasting accuracy

Fuzzy regression models have been applied to operational research (OR) applications such as forecasting. Some of previous studies on fuzzy regression analysis obtain crisp regression coefficients for eliminating the problem of increasing spreads for the estimated fuzzy responses as the magnitude of the independent variable increases; however, they still cannot cope with the situation of decreasing or variable spreads. This paper proposes a three-phase method to construct the fuzzy regression model with variable spreads to resolve this problem. In the first phase, on the basis of the extension principle, the membership functions of the least-squares estimates of regression coefficients are constructed to conserve completely the fuzziness of observations. In the second phase, then they are defuzzified by the center of gravity method to obtain crisp regression coefficients. In the third phase, the error terms of the proposed model are determined by setting each estimated spread equals its corresponding observed spread. Furthermore, the Mamdani fuzzy inference system is adopted for improving the accuracy of its forecasts. Compared to the previous studies, the results from five examples and an application example of Japanese house prices show that the proposed fuzzy linear regression model has higher explanatory power and forecasting performance.     

A novel fuzzy linear regression model based on a non-equality possibility index and optimum uncertainty

Various kinds of fuzzy regression models are introduced in the literature and many different methods are proposed to estimate fuzzy parameters of the models. In this study, a new approach is introduced to find the parameters of a linear fuzzy regression, with fuzzy outputs, the input data of which is measured by crisp numbers. Based on a non-equality possibility index, a new objective function is designed and solved, by which a minimum degree of acceptable uncertainty (the h-level or h-cut) is found. Four numerical examples are presented to compare the proposed approach with some other methods. Results show superiority of the new approach based on the criterion used by Kim and Bishu in the cases studied here. A realistic application of the proposed method is also presented, by which the total energy consumption of the Residential-Commercial sector in Iran is modeled using three variables of the GDP, number of the Households and an Energy Price index as inputs (exogenous variables) to the model.     

Fuzzy functions with support vector machines

A new fuzzy system modeling (FSM) approach that identifies the fuzzy functions using support vector machines (SVM) is proposed. This new approach is structurally different from the fuzzy rule base approaches and fuzzy regression methods. It is a new alternate version of the earlier FSM with fuzzy functions approaches. SVM is applied to determine the support vectors for each fuzzy cluster obtained by fuzzy c-means (FCM) clustering algorithm. Original input variables, the membership values obtained from the FCM together with their transformations form a new augmented set of input variables. The performance of the proposed system modeling approach is compared to previous fuzzy functions approaches, standard SVM, LSE methods using an artificial sparse dataset and a real-life non-sparse dataset. The results indicate that the proposed fuzzy functions with support vector machines approach is a feasible and stable method for regression problems and results in higher performances than the classical statistical methods.     

Fuzzy regression using least absolute deviation estimators

In fuzzy regression, that was first proposed by Tanaka et al. (Eur J Oper Res 40:389–396, 1989; Int Cong Appl Syst Cybern 4:2933–2938, 1980; IEEE Trans SystMan Cybern 12:903–907, 1982), there is a tendency that the greater the values of independent variables, the wider the width of the estimated dependent variables. This causes a decrease in the accuracy of the fuzzy regression model constructed by the least squares method. This paper suggests the least absolute deviation estimators to construct the fuzzy regression model, and investigates the performance of the fuzzy regression models with respect to a certain errormeasure. Simulation studies and examples show that the proposed model produces less error than the fuzzy regression model studied by many authors that use the least squares method when the data contains fuzzy outliers.     

Assessing airborne transmission of foot and mouth disease using fuzzy logic

In case of an outbreak of foot and mouth disease, the prediction of airborne spread is an important tool for decision-makers to assess the potential risk of secondary infections. Modelling approaches such as the Gaussian dispersion or Lagrangian particle model have been established but are complex to use and the structure of the models is fixed rather than adjustable to emerging disease situations. The aim of the present study was to evaluate the application of fuzzy logic as a modelling technique based on linguistic variables. Fuzzy logic models are easy to use and to modify. Adaptations to emerging outbreaks seem feasible. Using the Gaussian dispersion model as a reference, livestock-specific fuzzy logic models were developed. In a stepwise modelling process, the input parameters of the Gaussian model were added one-by-one into the fuzzy models. On the basis of weather data and randomly allocated farms, a validation dataset with 10,000 observations was generated and used in a 10-fold cross validation to compare the two modelling approaches. A good agreement between the Gaussian dispersion and the fuzzy logic models concerning the main directions of virus spread were found. The measure of agreement ranged between 87.0% and 99.9%. Falsely classified observations occurred mostly in proximity to the boundary of virus transmission based on the Gaussian dispersion model. In conclusion, fuzzy logic determined the same risk of infection for secondary cases than the Gaussian dispersion model. Limitations to certain livestock were not observed. The inclusion of up to four input variables did not influence the results in a mentionable amount. Including additional input variables into the fuzzy models could improve its application in assessing the risk of airborne foot and mouth disease transmission furthermore.     

Modeling manufacturing processes using a genetic programming-based fuzzy regression with detection of outliers

Fuzzy regression (FR) been demonstrated as a promising technique for modeling manufacturing processes where availability of data is limited. FR can only yield linear type FR models which have a higher degree of fuzziness, but FR ignores higher order or interaction terms and the influence of outliers, all of which usually exist in the manufacturing process data. Genetic programming (GP), on the other hand, can be used to generate models with higher order and interaction terms but it cannot address the fuzziness of the manufacturing process data. In this paper, genetic programming-based fuzzy regression (GP-FR), which combines the advantages of the two approaches to overcome the deficiencies of the commonly used existing modeling methods, is proposed in order to model manufacturing processes. GP-FR uses GP to generate model structures based on tree representation which can represent interaction and higher order terms of models, and it uses an FR generator based on fuzzy regression to determine outliers in experimental data sets. It determines the contribution and fuzziness of each term in the model by using experimental data excluding the outliers. To evaluate the effectiveness of GP-FR in modeling manufacturing processes, it was used to model a non-linear system and an epoxy dispensing process. The results were compared with those based on two commonly used FR methods, Tanka’s FR and Peters’ FR. The prediction accuracy of the models developed based on GP-FR was shown to be better than that of models based on the other two FR methods.     

The construction of fuzzy least squares estimators in fuzzy linear regression models

A new concept and method of imposing imprecise (fuzzy) input and output data upon the conventional linear regression model is proposed. Under the considerations of fuzzy parameters and fuzzy arithmetic operations (fuzzy addition and multiplication), we propose a fuzzy linear regression model which has the similar form as that of conventional one. We conduct the h-level (conventional) linear regression models of fuzzy linear regression model for the sake of invoking the statistical techniques in (conventional) linear regression analysis for real-valued data. In order to determine the sign (nonnegativity or nonpositivity) of fuzzy parameters, we perform the statistical testing hypotheses and evaluate the confidence intervals. Using the least squares estimators obtained from the h-level linear regression models, we can construct the membership functions of fuzzy least squares estimators via the form of “Resolution Identity” which is well-known in fuzzy sets theory. In order to obtain the membership degree of any given estimate taken from the fuzzy least squares estimator, optimization problems have to be solved. We also provide two computational procedures to deal with those optimization problems.     

Off-line signature verification and forgery detection using fuzzy modeling

Automatic signature verification is a well-established and an active area of research with numerous applications such as bank check verification, ATM access, etc. This paper proposes a novel approach to the problem of automatic off-line signature verification and forgery detection. The proposed approach is based on fuzzy modeling that employs the Takagi-Sugeno (TS) model. Signature verification and forgery detection are carried out using angle features extracted from box approach. Each feature corresponds to a fuzzy set. The features are fuzzified by an exponential membership function involved in the TS model, which is modified to include structural parameters. The structural parameters are devised to take account of possible variations due to handwriting styles and to reflect moods. The membership functions constitute weights in the TS model. The optimization of the output of the TS model with respect to the structural parameters yields the solution for the parameters. We have also derived two TS models by considering a rule for each input feature in the first formulation (Multiple rules) and by considering a single rule for all input features in the second formulation. In this work, we have found that TS model with multiple rules is better than TS model with single rule for detecting three types of forgeries; random, skilled and unskilled from a large database of sample signatures in addition to verifying genuine signatures. We have also devised three approaches, viz., an innovative approach and two intuitive approaches using the TS model with multiple rules for improved performance.     

Genetic algorithms-based fuzzy regression analysis

 This paper describes the concept of fuzzy regression analysis based on genetic algorithms. It is shown that the performance of fuzzy regression models may be improved and fuzzy modeling technique can be simplified by incorporating genetic algorithms into regression analysis procedure. The effectiveness of the proposed approach is illustrated through simulation of fuzzy linear regression model obtained by other authors and comparison of the results. The paper further demonstrates the applications of the approach to the manufacturing and business problems.     

Hybrid Fuzzy Modelling for Model Predictive Control

Model predictive control (MPC) has become an important area of research and is also an approach that has been successfully used in many industrial applications. In order to implement a MPC algorithm, a model of the process we are dealing with is needed. Due to the complex hybrid and nonlinear nature of many industrial processes, obtaining a suitable model is often a difficult task. In this paper a hybrid fuzzy modelling approach with a compact formulation is introduced. The hybrid system hierarchy is explained and the Takagi–Sugeno fuzzy formulation for the hybrid fuzzy modelling purposes is presented. An efficient method for identifying the hybrid fuzzy model is also proposed. A MPC algorithm suitable for systems with discrete inputs is treated. The benefits of the MPC algorithm employing the hybrid fuzzy model are verified on a batch-reactor simulation example: a comparison between the proposed modern intelligent (fuzzy) approach and a classic (linear) approach was made. It was established that the MPC algorithm employing the proposed hybrid fuzzy model clearly outperforms the approach where a hybrid linear model is used, which justifies the usability of the hybrid fuzzy model. The hybrid fuzzy formulation introduces a powerful model that can faithfully represent hybrid and nonlinear dynamics of systems met in industrial practice, therefore, this approach demonstrates a significant advantage for MPC resulting in a better control performance.     

Development of PCA-based cluster quantile regression (PCA-CQR) framework for streamflow prediction: Application to the Xiangxi river watershed,China

In this study, a PCA-based cluster quantile regression (PCA-CQR) method was proposed through integrating principal component analysis and quantile regression approaches into a stepwise cluster analysis framework. In detail, the principal component analysis was adopted to overcome the multicollinearity among the explanatory variables, while the quantile regression approach was used to provide probabilistic information in prediction. The proposed PCA-CQR method can effectively capture discrete and nonlinear relationships between explanatory and response variables. The applicability of PCA-CQR was demonstrated by a case study of monthly streamflow prediction in the Xiangxi River, China. The nonlinearity between the hydro-meteorological variables and the streamflow measurements was characterized through the measure of maximal information coefficient (MIC), which demonstrated the need of the proposed PCA-CQR method. The results showed that the previous monthly streamflow and precipitation, as well as potential evapotranspiration in current month posed significant nonlinear impacts on the streamflow in current month. Three components could well reflect the total variance of the input variables. Comparison between traditional SCA and PCA-CQR showed that the proposed approach could provide more accurate predictions than traditional SCA methods. Moreover, probabilistic forecasts could be provided by PCA-CQR, and the 90% predictive intervals could well bracket the observations in both calibration and validation periods. Also, sensitivity analysis was performed to identify the impacts of the control parameters in PCA-CQR on the performance of the proposed model. The results showed the proposed PCA-CQR improved the robustness of traditional SCA. Finally, comparison among PCA-CQR, GRNN and MLR also showed the effectiveness of the proposed method.     

A fuzzy regression model based on distances and random variables with crisp input and fuzzy output data: a case study in biomass production

Least-squares technique is well-known and widely used to determine the coefficients of a explanatory model from observations based on a concept of distance. Traditionally, the observations consist of pairs of numeric values. However, in many real-life problems, the independent or explanatory variable can be observed precisely (for instance, the time) and the dependent or response variable is usually described by approximate values, such as “about £300\pounds300” or “approximately $500”, instead of exact values, due to sources of uncertainty that may affect the response. In this paper, we present a new technique to obtain fuzzy regression models that consider triangular fuzzy numbers in the response variable. The procedure solves linear and non-linear problems and is easy to compute in practice and may be applied in different contexts. The usefulness of the proposed method is illustrated using simulated and real-life examples.     

An inferential modeling method using enumerative PLS based nonnegative garrote regression

In this paper a robust linear regression method with variable selection is proposed for predicting desirable end-of-line quality variables in complex industrial processes. The development of such prediction models is challenging because there is usually a large pool of candidate explanatory variables, limited sample data, and multicollinearity among explanatory variables. The proposed method is named as the enumerative partial least square based nonnegative garrote regression. It employs partial least square regression in enumerative manner to generate initial model coefficients and then uses a nonnegative garrote method to shrink original coefficients so that irrelevant variables can be eliminated implicitly. Analysis about the advantages of the proposed method is provided compared to existing state-of-art model construction methods. Two simulation examples as well as an industrial application in a local semiconductor factory unit are used to validate the proposed method. These examples witness substantial improvement in terms of accuracy and robustness in variable selection compared to existing methods. Specifically, for the industrial case the percentages of improvement in terms of root mean squared error is up to 24.3% compared with the previous work.