Robust mixture modelling using the t distribution

Normal mixture models are being increasingly used to model the distributions of a wide variety of random phenomena and to cluster sets of continuous multivariate data. However, for a set of data containing a group or groups of observations with longer than normal tails or atypical observations, the use of normal components may unduly affect the fit of the mixture model. In this paper, we consider a more robust approach by modelling the data by a mixture of t distributions. The use of the ECM algorithm to fit this t mixture model is described and examples of its use are given in the context of clustering multivariate data in the presence of atypical observations in the form of background noise.     

Mixture of linear mixed models using multivariate t distribution

Linear mixed models are widely used when multiple correlated measurements are made on each unit of interest. In many applications, the units may form several distinct clusters, and such heterogeneity can be more appropriately modelled by a finite mixture linear mixed model. The classical estimation approach, in which both the random effects and the error parts are assumed to follow normal distribution, is sensitive to outliers, and failure to accommodate outliers may greatly jeopardize the model estimation and inference. We propose a new mixture linear mixed model using multivariate t distribution. For each mixture component, we assume the response and the random effects jointly follow a multivariate t distribution, to conveniently robustify the estimation procedure. An efficient expectation conditional maximization algorithm is developed for conducting maximum likelihood estimation. The degrees of freedom parameters of the t distributions are chosen data adaptively, for achieving flexible trade-off between estimation robustness and efficiency. Simulation studies and an application on analysing lung growth longitudinal data showcase the efficacy of the proposed approach.     

A dynamic EM algorithm for estimating mixture proportions

We investigate the use of a dynamic form of the EM algorithm to estimate proportions in finite mixtures of known distributions. We prove a consistency result for this algorithm, which employs only a single EM update for each new observation. Our aim is to demonstrate that the slow convergence rate of the EM algorithm in many applications is of little practical consequence in a situation when data is frequently being updated.     

Bayesian analysis of hierarchical linear mixed modeling using the multivariate t distribution

This article presents a fully Bayesian approach to modeling incomplete longitudinal data using the t linear mixed model with AR(p) dependence. Markov chain Monte Carlo (MCMC) techniques are implemented for computing posterior distributions of parameters. To facilitate the computation, two types of auxiliary indicator matrices are incorporated into the model. Meanwhile, the constraints on the parameter space arising from the stationarity conditions for the autoregressive parameters are handled by a reparametrization scheme. Bayesian predictive inferences for the future vector are also investigated. An application is illustrated through a real example from a multiple sclerosis clinical trial.     

A semiparametric approach for modelling multivariate nonlinear time series

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach. The Canadian Journal of Statistics 47: 668–687; 2019 © 2019 Statistical Society of Canada     

Objective Bayesian Analysis of Skew‐t Distributions

Abstract. We study the Jeffreys prior and its properties for the shape parameter of univariate skew‐t distributions with linear and nonlinear Student's t skewing functions. In both cases, we show that the resulting priors for the shape parameter are symmetric around zero and proper. Moreover, we propose a Student's t approximation of the Jeffreys prior that makes an objective Bayesian analysis easy to perform. We carry out a Monte Carlo simulation study that demonstrates an overall better behaviour of the maximum a posteriori estimator compared with the maximum likelihood estimator. We also compare the frequentist coverage of the credible intervals based on the Jeffreys prior and its approximation and show that they are similar. We further discuss location‐scale models under scale mixtures of skew‐normal distributions and show some conditions for the existence of the posterior distribution and its moments. Finally, we present three numerical examples to illustrate the implications of our results on inference for skew‐t distributions.     

Implementing particle swarm optimization algorithm to estimate the mixture of two Weibull parameters with censored data

We present the maximum likelihood estimation (MLE) via particle swarm optimization (PSO) algorithm to estimate the mixture of two Weibull parameters with complete and multiple censored data. A simulation study is conducted to assess the performance of the MLE via PSO algorithm, quasi-Newton method and expectation-maximization (EM) algorithm for different parameter settings and sample sizes in both uncensored and censored cases. The simulation results showed that the PSO algorithm outperforms the quasi-Newton method and the EM algorithm in most cases regarding bias and root mean square errors. Two numerical examples are used to demonstrate the performance of our proposed method.     

Estimating finite mixture of continuous trees using penalized mutual information

Abstract

In this paper we introduce continuous tree mixture model that is the mixture of undirected graphical models with tree structured graphs and is considered as multivariate analysis with a non parametric approach. We estimate its parameters, the component edge sets and mixture proportions through regularized maximum likalihood procedure. Our new algorithm, which uses expectation maximization algorithm and the modified version of Kruskal algorithm, simultaneosly estimates and prunes the mixture component trees. Simulation studies indicate this method performs better than the alternative Gaussian graphical mixture model. The proposed method is also applied to water-level data set and is compared with the results of Gaussian mixture model.     

Robust mixture modeling using the skew <Emphasis Type="Italic">t</Emphasis> distribution

A finite mixture model using the Student's t distribution has been recognized as a robust extension of normal mixtures. Recently, a mixture of skew normal distributions has been found to be effective in the treatment of heterogeneous data involving asymmetric behaviors across subclasses. In this article, we propose a robust mixture framework based on the skew t distribution to efficiently deal with heavy-tailedness, extra skewness and multimodality in a wide range of settings. Statistical mixture modeling based on normal, Student's t and skew normal distributions can be viewed as special cases of the skew t mixture model. We present analytically simple EM-type algorithms for iteratively computing maximum likelihood estimates. The proposed methodology is illustrated by analyzing a real data example.     

Estimation of parameters of the inverse gaussian-weibull mixture model

The main objective of this work is to estimate the 5-dime-nsional vector of parameters (p,μ,λ,α,c) of the mixture of an Inverse Gaussian IG(μ,λ) and Weibull W(α,c) distributions with mixing proportion p. We use the maximum Likelihood method (MLM) and the weighted maximum likelihood method (WMLM), both under the sampling schemes suggested by Hosmer (1973). Simulation study shows that the WMLM performs best, when Hosmer's model 2 is used, in the sense of minimizing the mean square error.     

A finite mixture approach to joint clustering of individuals and multivariate discrete outcomes

In this work, we modify finite mixtures of factor analysers to provide a method for simultaneous clustering of subjects and multivariate discrete outcomes. The joint clustering is performed through a suitable reparameterization of the outcome (column)-specific parameters. We develop an expectation–maximization-type algorithm for maximum likelihood parameter estimation where the maximization step is divided into orthogonal sub-blocks that refer to row and column-specific parameters, respectively. Model performance is evaluated via a simulation study with varying sample size, number of outcomes and row/column-specific clustering (partitions). We compare the performance of our model with the performance of standard model-based biclustering approaches. The proposed method is also demonstrated on a benchmark data set where a multivariate binary response is considered.     

A mixture of the manova and gmanova models

A mixture of the MANOVA and GMANOVA models is presented. The expected value of the response matrix in this model is the sum of two matrix components. The first component represents the GMANOVA portion and the second component represents the MANOVA portion. Maximum likelihood estimators are derived for the parameters in this model, and goodness-of-fit tests are constructed for fuller models via the likelihood ration criterion. Finally, likelihood ration tests for general liinear hypotheses are developed and a numerical example is presented.     

Bayesian item response model: a generalized approach for the abilities' distribution using mixtures

Traditional Item Response Theory models assume the distribution of the abilities of the population in study to be Gaussian. However, this may not always be a reasonable assumption, which motivates the development of more general models. This paper presents a generalized approach for the distribution of the abilities in dichotomous 3-parameter Item Response models. A mixture of normal distributions is considered, allowing for features like skewness, multimodality and heavy tails. A solution is proposed to deal with model identifiability issues without compromising the flexibility and practical interpretation of the model. Inference is carried out under the Bayesian Paradigm through a novel MCMC algorithm. The algorithm is designed in a way to favour good mixing and convergence properties and is also suitable for inference in traditional IRT models. The efficiency and applicability of our methodology is illustrated in simulated and real examples.     

The approximate null distribution of the likelihood ratio test for a mixture of two bivariate normal distributions with equal co variance

We define the mixture likelihood approach to clustering by discussing the sampling distribution of the likelihood ratio test of the null hypothesis that we have observed a sample of observations of a variable having the bivariate normal distribution versus the alternative that the variable has the bivariate normal mixture with unequal means and common within component covariance matrix. The empirical distribution of the likelihood ratio test indicates that convergence to the chi-squared distribution with 2 df is at best very slow, that the sample size should be 5000 or more for the chi-squared result to hold, and that for correlations between 0.1 and 0.9 there is little, if any, dependence of the null distribution on the correlation. Our simulation study suggests a heuristic function based on the gamma.     

Estimation of principal points for a multivariate binary distribution using a log-linear model

This article proposes a method for estimating principal points for a multivariate binary distribution, assuming a log-linear model for the distribution. Through numerical simulation studies, the proposed parametric estimation method using a log-linear model is compared with a nonparametric estimation method.     

Robust variable selection in finite mixture of regression models using the t distribution

Abstract

Variable selection in finite mixture of regression (FMR) models is frequently used in statistical modeling. The majority of applications of variable selection in FMR models use a normal distribution for regression error. Such assumptions are unsuitable for a set of data containing a group or groups of observations with heavy tails and outliers. In this paper, we introduce a robust variable selection procedure for FMR models using the t distribution. With appropriate selection of the tuning parameters, the consistency and the oracle property of the regularized estimators are established. To estimate the parameters of the model, we develop an EM algorithm for numerical computations and a method for selecting tuning parameters adaptively. The parameter estimation performance of the proposed model is evaluated through simulation studies. The application of the proposed model is illustrated by analyzing a real data set.     

On the distribution of hotelling's t2 and multiple correlation r2when sampling from a mixture of two normals

In this paper the non-null distribution of Hotelling's T² and the null distribution of multiple correlation R² are derived when the sample is taken from a mixture of two p-component multivariate normal distributions with mean vectors μ₁ and μ₂ respectively and common covariance matrix ∑, ∑. In a special case the non-null distribution of R² is a l s o given, while the general noncentral distribution is given i n Awan (1981). These results have been used to study the robustness of T² and R² tests by Srivastava and Awan (1982), and Awan and Srivastava (1982) respectively.     

分层线性模型的最大后验估计

最大后验估计（MAPE）和最大似然估计（MLE）都是重要的参数点估计方法。在介绍一般分层线性模型（HLM）MAPE方法的基础上,给出这种方法的期望最大化算法（EM）的具体步骤,运用对数似然函数的二阶导数推导了MAPE估计的方差估计量。同时运用数据模拟比较了EM算法下的MAPE和MLE。对于固定效应的估计,两种方法得到的估计量是一致的。当组数较少时,EM计算的MAPE的方差协方差成分比MLE的更靠近真实值,而且MAPE的迭代次数明显小于MLE。