共查询到20条相似文献,搜索用时 156 毫秒
1.
This paper presents a novel knowledge-based linear classification model for multi-category discrimination of sets or objects with prior knowledge. The prior knowledge is in the form of multiple polyhedral sets belonging to one or more categories or classes and it is introduced as additional constraints into the formulation of the Tikhonov linear least squares multi-class support vector machine model. The resulting formulation leads to a least squares problem that can be solved using matrix methods or iterative methods. Investigations include the development of a linear knowledge-based classification model extended to the case of multi-categorical discrimination and expressed as a single unconstrained optimization problem. Advantages of this formulation include explicit expressions for the classification weights of the classifier(s) and its ability to incorporate and handle prior knowledge directly to the classifiers. In addition it can provide fast solutions to the optimal classification weights for multi-categorical separation without the use of specialized solver-software. To evaluate the model, data and prior knowledge from the Wisconsin breast cancer prognosis and two-phase flow regimes in pipes were used to train and test the proposed formulation. 相似文献
2.
Theodore B. Trafalis Olutayo O. Oladunni Michael B. Richman 《Computational Management Science》2011,8(3):281-297
A knowledge-based linear Tihkonov regularization classification model for tornado discrimination is presented. Twenty-three
attributes, based on the National Severe Storms Laboratory’s Mesoscale Detection Algorithm, are used as prior knowledge. Threshold
values for these attributes are employed to discriminate the data into two classes (tornado, non-tornado). The Weather Surveillance
Radar 1998 Doppler is used as a source of data streaming every 6 min. The combination of data and prior knowledge is used
in the development of a least squares problem that can be solved using matrix or iterative methods. Advantages of this formulation
include explicit expressions for the classification weights of the classifier and its ability to incorporate and handle prior
knowledge directly to the classifiers. Comparison of the present approach to that of Fung et al. [in Proceedings neural information
processing systems (NIPS 2002), Vancouver, BC, December 10–12, 2002], over a suite of forecast evaluation indices, demonstrates
that the Tikhonov regularization model is superior for discriminating tornadic from non-tornadic storms. 相似文献
3.
Classification is a main data mining task, which aims at predicting the class label of new input data on the basis of a set of pre-classified samples. Multiple criteria linear programming (MCLP) is used as a classification method in the data mining area, which can separate two or more classes by finding a discriminate hyperplane. Although MCLP shows good performance in dealing with linear separable data, it is no longer applicable when facing with nonlinear separable problems. A kernel-based multiple criteria linear programming (KMCLP) model is developed to solve nonlinear separable problems. In this method, a kernel function is introduced to project the data into a higher-dimensional space in which the data will have more chance to be linear separable. KMCLP performs well in some real applications. However, just as other prevalent data mining classifiers, MCLP and KMCLP learn only from training examples. In the traditional machine learning area, there are also classification tasks in which data sets are classified only by prior knowledge, i.e. expert systems. Some works combine the above two classification principles to overcome the faults of each approach. In this paper, we provide our recent works which combine the prior knowledge and the MCLP or KMCLP model to solve the problem when the input consists of not only training examples, but also prior knowledge. Specifically, how to deal with linear and nonlinear knowledge in MCLP and KMCLP models is the main concern of this paper. Numerical tests on the above models indicate that these models are effective in classifying data with prior knowledge. 相似文献
4.
《Applied Mathematics Letters》2007,20(1):82-87
In this work, we study and analyze the regularized weighted total least squares (RWTLS) formulation. Our regularization of the weighted total least squares problem is based on the Tikhonov regularization. Numerical examples are presented to demonstrate the effectiveness of the RWTLS method. 相似文献
5.
应用SAS解非线性回归问题 总被引:2,自引:0,他引:2
.应用SAS/STAT估计非线性回归模型中的参数.首先,通过变量代换,把可以线性化的非线性回归模型化为线性回归模型,并用普通最小二乘法、主成分分析法和偏最小二乘法求模型中的参数和回归模型.其次,通过改良的高斯—牛顿迭代法来估计Logistic模型和Compertz模型中的参数. 相似文献
6.
Sangheum Hwang Dohyun Kim Myong K Jeong Bong-Jin Yum 《The Journal of the Operational Research Society》2015,66(8):1385-1398
The kernel-based regression (KBR) method, such as support vector machine for regression (SVR) is a well-established methodology for estimating the nonlinear functional relationship between the response variable and predictor variables. KBR methods can be very sensitive to influential observations that in turn have a noticeable impact on the model coefficients. The robustness of KBR methods has recently been the subject of wide-scale investigations with the aim of obtaining a regression estimator insensitive to outlying observations. However, existing robust KBR (RKBR) methods only consider Y-space outliers and, consequently, are sensitive to X-space outliers. As a result, even a single anomalous outlying observation in X-space may greatly affect the estimator. In order to resolve this issue, we propose a new RKBR method that gives reliable result even if a training data set is contaminated with both Y-space and X-space outliers. The proposed method utilizes a weighting scheme based on the hat matrix that resembles the generalized M-estimator (GM-estimator) of conventional robust linear analysis. The diagonal elements of hat matrix in kernel-induced feature space are used as leverage measures to downweight the effects of potential X-space outliers. We show that the kernelized hat diagonal elements can be obtained via eigen decomposition of the kernel matrix. The regularized version of kernelized hat diagonal elements is also proposed to deal with the case of the kernel matrix having full rank where the kernelized hat diagonal elements are not suitable for leverage. We have shown that two kernelized leverage measures, namely, the kernel hat diagonal element and the regularized one, are related to statistical distance measures in the feature space. We also develop an efficiently kernelized training algorithm for the parameter estimation based on iteratively reweighted least squares (IRLS) method. The experimental results from simulated examples and real data sets demonstrate the robustness of our proposed method compared with conventional approaches. 相似文献
7.
Adi Ben-Israel 《Israel Journal of Mathematics》1967,5(4):211-224
Nonlinear least squares problems over convex sets inR
n are treated here by iterative methods which extend the classical Newton, gradient and steepest descent methods and the methods
studied recently by Pereyra and the author. Applications are given to nonlinear least squares problems under linear constraint,
and to linear and nonlinear inequalities.
Part of the research underlying this report was undertaken for the Office of Naval Research, Contract Nonr-1228(10), Project
NR047-021, and for the U.S. Army Research Office — Durham, Contract No. DA-31-124-ARO-D-322 at Northwestern University. Reproduction
of this paper in whole or in part is permitted for any purpose of the United States Government. 相似文献
8.
V. V. Volkov V. I. Erokhin V. V. Kakaev A. Yu. Onufrei 《Computational Mathematics and Mathematical Physics》2017,57(9):1416-1426
Tikhonov’s regularized method of least squares and its generalizations to non-Euclidean norms, including polyhedral, are considered. The regularized method of least squares is reduced to mathematical programming problems obtained by “instrumental” generalizations of the Tikhonov lemma on the minimal (in a certain norm) solution of a system of linear algebraic equations with respect to an unknown matrix. Further studies are needed for problems concerning the development of methods and algorithms for solving reduced mathematical programming problems in which the objective functions and admissible domains are constructed using polyhedral vector norms. 相似文献
9.
This paper extends prior work by the authors on solving nonlinear least squares unconstrained problems using a factorized
quasi-Newton technique. With this aim we use a primal-dual interior-point algorithm for nonconvex nonlinear programming. The
factorized quasi-Newton technique is now applied to the Hessian of the Lagrangian function for the transformed problem which
is based on a logarithmic barrier formulation. We emphasize the importance of establishing and maintaining symmetric quasi-definiteness
of the reduced KKT system. The algorithm then tries to choose a step size that reduces a merit function, and to select a penalty
parameter that ensures descent directions along the iterative process. Computational results are included for a variety of
least squares constrained problems and preliminary numerical testing indicates that the algorithm is robust and efficient
in practice. 相似文献
10.
《Optimization》2012,61(12):1467-1490
Large outliers break down linear and nonlinear regression models. Robust regression methods allow one to filter out the outliers when building a model. By replacing the traditional least squares criterion with the least trimmed squares (LTS) criterion, in which half of data is treated as potential outliers, one can fit accurate regression models to strongly contaminated data. High-breakdown methods have become very well established in linear regression, but have started being applied for non-linear regression only recently. In this work, we examine the problem of fitting artificial neural networks (ANNs) to contaminated data using LTS criterion. We introduce a penalized LTS criterion which prevents unnecessary removal of valid data. Training of ANNs leads to a challenging non-smooth global optimization problem. We compare the efficiency of several derivative-free optimization methods in solving it, and show that our approach identifies the outliers correctly when ANNs are used for nonlinear regression. 相似文献
11.
《Applied Mathematical Modelling》2005,29(7):673-689
In this study a new insight into least squares regression is identified and immediately applied to estimating the parameters of nonlinear rational models. From the beginning the ordinary explicit expression for linear in the parameters model is expanded into an implicit expression. Then a generic algorithm in terms of least squares error is developed for the model parameter estimation. It has been proved that a nonlinear rational model can be expressed as an implicit linear in the parameters model, therefore, the developed algorithm can be comfortably revised for estimating the parameters of the rational models. The major advancement of the generic algorithm is its conciseness and efficiency in dealing with the parameter estimation problems associated with nonlinear in the parameters models. Further, the algorithm can be used to deal with those regression terms which are subject to noise. The algorithm is reduced to an ordinary least square algorithm in the case of linear or linear in the parameters models. Three simulated examples plus a realistic case study are used to test and illustrate the performance of the algorithm. 相似文献
12.
Circle fitting by linear and nonlinear least squares 总被引:2,自引:0,他引:2
I. D. Coope 《Journal of Optimization Theory and Applications》1993,76(2):381-388
The problem of determining the circle of best fit to a set of points in the plane (or the obvious generalization ton-dimensions) is easily formulated as a nonlinear total least-squares problem which may be solved using a Gauss-Newton minimization algorithm. This straight-forward approach is shown to be inefficient and extremely sensitive to the presence of outliers. An alternative formulation allows the problem to be reduced to a linear least squares problem which is trivially solved. The recommended approach is shown to have the added advantage of being much less sensitive to outliers than the nonlinear least squares approach.This work was completed while the author was visiting the Numerical Optimisation Centre, Hatfield Polytechnic and benefitted from the encouragement and helpful suggestions of Dr. M. C. Bartholomew-Biggs and Professor L. C. W. Dixon. 相似文献
13.
Marcelo Espinoza Johan A. K. Suykens Bart De Moor 《Computational Management Science》2006,3(2):113-129
Based on the Nyström approximation and the primal-dual formulation of the least squares support vector machines, it becomes possible to apply a nonlinear model to a large scale regression problem. This is done by using a sparse approximation of the nonlinear mapping induced by the kernel matrix, with an active selection of support vectors based on quadratic Renyi entropy criteria. The methodology is applied to the case of load forecasting as an example of a real-life large scale problem in industry. The forecasting performance, over ten different load series, shows satisfactory results when the sparse representation is built with less than 3% of the available sample. 相似文献
14.
Aiming at identifying nonlinear systems, one of the most challenging problems in system identification, a class of data-driven recursive least squares algorithms are presented in this work. First, a full form dynamic linearization based linear data model for nonlinear systems is derived. Consequently, a full form dynamic linearization-based data-driven recursive least squares identification method for estimating the unknown parameter of the obtained linear data model is proposed along with convergence analysis and prediction of the outputs subject to stochastic noises. Furthermore, a partial form dynamic linearization-based data-driven recursive least squares identification algorithm is also developed as a special case of the full form dynamic linearization based algorithm. The proposed two identification algorithms for the nonlinear nonaffine discrete-time systems are flexible in applications without relying on any explicit mechanism model information of the systems. Additionally, the number of the parameters in the obtained linear data model can be tuned flexibly to reduce computation complexity. The validity of the two identification algorithms is verified by rigorous theoretical analysis and simulation studies. 相似文献
15.
Alexander Goldenshluger 《Journal of multivariate analysis》2003,84(1):40-60
The problem of optimal prediction in the stochastic linear regression model with infinitely many parameters is considered. We suggest a prediction method that outperforms asymptotically the ordinary least squares predictor. Moreover, if the random errors are Gaussian, the method is asymptotically minimax over ellipsoids in ?2. The method is based on a regularized least squares estimator with weights of the Pinsker filter. We also consider the case of dynamic linear regression, which is important in the context of transfer function modeling. 相似文献
16.
Time series data with periodic trends like daily temperatures or sales of seasonal products can be seen in periods fluctuating between highs and lows throughout the year. Generalized least squares estimators are often computed for such time series data as these estimators have minimum variance among all linear unbiased estimators. However, the generalized least squares solution can require extremely demanding computation when the data is large. This paper studies an efficient algorithm for generalized least squares estimation in periodic trended regression with autoregressive errors. We develop an algorithm that can substantially simplify generalized least squares computation by manipulating large sets of data into smaller sets. This is accomplished by coining a structured matrix for dimension reduction. Simulations show that the new computation methods using our algorithm can drastically reduce computing time. Our algorithm can be easily adapted to big data that show periodic trends often pertinent to economics, environmental studies, and engineering practices. 相似文献
17.
Jun-Feng Yin 《Applied mathematics and computation》2009,215(8):3007-3016
We propose a sparse approximate inverse preconditioner based on the Sherman-Morrison formula for Tikhonov regularized least square problems. Theoretical analysis shows that, the factorization method can take the advantage of the symmetric property of the coefficient matrix and be implemented cheaply. Combined with dropping rules, the incomplete factorization leads to a preconditioner for Krylov iterative methods to solve regularized least squares problems. Numerical experiments show that our preconditioner is competitive compared to existing methods, especially for ill-conditioned and rank deficient least squares problems. 相似文献
18.
A computational approach for solving regularized total least squares problems via a sequence of quadratic eigenvalue problems has recently been introduced by Sima, Van Huffel, and Golub. Combining this approach with the nonlinear Arnoldi method and reusing information from all previous quadratic eigenvalue problems, together with an early update of search spaces we arrive at a very efficient method for large regularized total least squares problems. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim) 相似文献
19.
Nonlinear least squares optimization problems in which the parameters can be partitioned into two sets such that optimal estimates
of parameters in one set are easy to solve for given fixed values of the parameters in the other set are common in practice.
Particularly ubiquitous are data fitting problems in which the model function is a linear combination of nonlinear functions,
which may be addressed with the variable projection algorithm due to Golub and Pereyra. In this paper we review variable projection,
with special emphasis on its application to matrix data. The generalization of the algorithm to separable problems in which
the linear coefficients of the nonlinear functions are subject to constraints is also discussed. Variable projection has been
instrumental for model-based data analysis in multi-way spectroscopy, time-resolved microscopy and gas or liquid chromatography
mass spectrometry, and we give an overview of applications in these domains, illustrated by brief case studies. 相似文献
20.