Ill-conditioned covariance create

http://www.mathworks.com/matlabcentral/answers/100210-why-do-i-receive-an-error-while-trying-to-generate-the-gaussian-mixture-parameter-estimates-from-a-d

http://www.cnblogs.com/tadoo/archive/2011/06/01/2065244.html

Error using gmcluster (line 180)
Ill-conditioned covariance created at iteration 2.

Error in gmdistribution.fit (line 174)
[S,NlogL,optimInfo] =...

 

This error message is expected because the GMDISTRIBUTION.FIT function is not able to fit a 2 component Gaussian mixture model to the dataset in Statistics Toolbox 7.1 (R2009a).

As a general guideline to Gaussian Mixture Model(GMM) fitting, the GMDISTRIBUTION function may converge to a solution where one or more of the components has an ill-conditioned or singular covariance matrix.

The following issues may result in an ill-conditioned covariance matrix:

1. The number of dimensions of your data is relatively high and there are not enough observations.

2. Some of the features (variables) of your data are highly correlated.

3. Some or all the features are discrete.

4. You tried to fit the data to too many components.

In general, you can avoid getting ill-conditioned covariance matrices by using one of the following precautions:

1.Pre-process your data to remove correlated features.

2.Set 'SharedCov' to true to use an equal covariance matrix for every component.

3.Set 'CovType' to 'diagonal'.

4.Use 'Regularize' to add a very small positive number to the diagonal of every covariance matrix.

5.Try another set of initial values.

 

以上是matlab给出的处理 singular 情况的方法，能解决大部分 singular 的问题，但不推荐随便采用，需要了解这样处理的含义。

推荐：

http://freemind.pluskid.org/machine-learning/regularized-gaussian-covariance-estimation/