Machine Learning (17) — GMM Algorithm

Preface: A simple application of the EM algorithm.

Algorithm workflow

Example: measure height of 1000 users; if male and female heights are N(μ₁,σ₁) and N(μ₂,σ₂) and groups are known, use MLE. If mixed and unlabeled, plain MLE fails—use GMM.

GMM (Gaussian Mixture Model): k Gaussian components linearly mixed; models data distribution; k often equals cluster count.

Mixture density:

PDF:

Log-likelihood:

E-step:

M-step:

Iterate until convergence.

Height/weight example

1. Imports

# Data, GMM, train_test_split, plotting

import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.colors
import matplotlib.pyplot as plt

from sklearn.mixture import GaussianMixture
from sklearn.model_selection import train_test_split
mpl.rcParams['font.sans-serif'] = [u'SimHei']
mpl.rcParams['axes.unicode_minus'] = False

2. Load data

data = pd.read_csv('datas/HeightWeight.csv')
print ("数据样本数量:%d, 特征数量:%d" % data.shape)
data_x = data[data.columns[1:]]
data_y = data[data.columns[0]]
data.head()

Preview:

3. Train/test split

x, x_test, y, y_test = train_test_split(data_x, data_y, train_size=0.6, random_state=0)

4. Train

gmm = GaussianMixture(n_components=2, covariance_type='full', random_state=28)
gmm.fit(x, y)

Model:

5. Parameters

print ('均值 = \n', gmm.means_)
print ('方差 = \n', gmm.covariances_)

6. Evaluation

y_hat = gmm.predict(x)
y_test_hat = gmm.predict(x_test)

change = (gmm.means_[0][0] > gmm.means_[1][0])
if change:
    z = y_hat == 0
    y_hat[z] = 1
    y_hat[~z] = 0
    z = y_test_hat == 0
    y_test_hat[z] = 1
    y_test_hat[~z] = 0

acc = np.mean(y_hat.ravel() == y.ravel())
acc_test = np.mean(y_test_hat.ravel() == y_test.ravel())
acc_str = u'训练集准确率：%.2f%%' % (acc * 100)
acc_test_str = u'测试集准确率：%.2f%%' % (acc_test * 100)
print (acc_str)
print (acc_test_str)