Gaussian mixture models: constrained and penalized approaches

The finite mixture of Gaussians is a well known model frequently used to classify a sample of observations. It is based on the idea of considering the observations as drawn by an heterogenous population where each sub-population is described by a component of the mixture. However, in practical applications the maximum likelihood estimation of model parameters poses several problems because the likelihood is unbounded and can be characterized by numerous spurios maxima. Several methods have been proposed in order to circumvent this problem, mainly based on the shrinking of the covariance matrices towards a prespicified matrix Ψ. In this work we consider two of those methods and investigate the problem of how to estimate Ψ from the data when a priori information is not available.