1.李航机器学习-多项式回归公式推导及python实现

多项式回归

项目链接：https://github.com/Wchenguang/gglearn/blob/master/PolynomialClassifier/李航机器学习-讲解/PolynomialClassifier.ipynb

公式推导

• 损失函数定义为平方损失函数

$$\begin{array}{l}\cos t=\frac{1}{2}(f(X)-Y{)}^2\\ L=\frac{1}{N}\times {\sum }_{i=1}^N\frac{1}{2}{\left({\sum }_{j=0}^M\left(w_j{x}_i^j\right)-y_i\right)}^2\end{array}$$

• 求导并使导数为0，直接求出权值

$$\begin{array}{c}\begin{array}{l}\frac{\partial L}{\partial w_k}=\frac{1}{N}\times {\sum }_{i=1}^N\frac{1}{2}\times 2\left(x_i^k\times \left({\sum }_{j=0}^M\left(w_j{x}_i^k\right)-y_i\right)\right)\\ =\frac{1}{N}\times {\sum }_{i=1}^N\left(x_i^k\times {\sum }_{j=0}^M\left(w_j{x}_i^j\right)-x_i^k\times y_i\right)\\ =0\end{array}\\ \downarrow \\ x_1^k{\sum }_{j=1}^m{w}_j{x}_1^j+x_2^K{\sum }_{j=1}^m{w}_j{x}_2^j\cdots x_n^K{\sum }_{j=1}^m{w}_j{x}_n^j={\sum }_{i=1}^n{x}_i^k{y}_i\\ \downarrow \\ \left[x_1^k\dots x_n^k\right]\left[\begin{array}{ccc}{x}_1^0& \cdots & x_1^m\\ & \cdots & \\ & ⋮& \\ x_n^0& \cdots & x_n^m\end{array}\right]\left[\begin{array}{c}{w}_0\\ ⋮\\ w_m\end{array}\right]=\left[x_1^k\cdots x_n^k\right]\left[\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right]\end{array}$$

• 上式所求的$w_k$可以用含有$w_0{w}_1\cdots w_{k-1}{w}_{k+1}\cdots w_m$ 的式子表示，这些$w$值表示的是相应权重值的最优解，因而可以利用线性代数将所有权重值求出

$$\begin{array}{c}\left[x_1^k\dots x_n^k\right]\left[\begin{array}{ccc}{x}_1^0& \cdots & x_1^m\\ & \cdots & \\ & ⋮& \\ x_n^0& \cdots & x_n^m\end{array}\right]\left[\begin{array}{c}{w}_0\\ ⋮\\ w_m\end{array}\right]=\left[x_1^k\cdots x_n^k\right]\left[\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right]\\ \downarrow \\ \left[\begin{array}{ccc}{x}_1^0& \cdots & x_n^0\\ & \cdots & \\ & ⋮& \\ x_1^m& \cdots & x_n^m\end{array}\right]\left[\begin{array}{ccc}{x}_1^0& \cdots & x_1^m\\ & \cdots & \\ & ⋮& \\ x_n^0& \cdots & x_n^m\end{array}\right]\left[\begin{array}{c}{w}_0\\ ⋮\\ w_m\end{array}\right]=\left[\begin{array}{ccc}{x}_1^0& \cdots & x_n^0\\ & \cdots & \\ & ⋮& \\ x_1^m& \cdots & x_n^m\end{array}\right]\left[\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right]\end{array}$$

• 由上式即可得梯度下降中的正规方程
  $$\begin{array}{c}{x}^{\top }xw=x^{\top }y\\ \downarrow \\ w={\left(x^{\top }x\right)}^{-1}{x}^{\top }y\end{array}$$

实现多项式分类器PolynomialRegression

import numpy as np
class PolynomialRegression:
    '''
    只支持numpy数组的输入
    '''
    def __init__(self):
        pass
    def fit(self, x, y):
        '''
        只支持二维数组
        '''
        x= np.hstack((np.ones((len(x), 1)), x))
        x = np.mat(x)
        y = np.mat(y)
        self.w = np.mat((x.T * x).I * x.T * y)
        return self
    def transform(self, x):
        x= np.hstack((np.ones((len(x), 1)), x))
        x = np.mat(x)
        return x * self.w
    def fit_transform(self, x, y):
        x= np.hstack((np.ones((len(x), 1)), x))
        x = np.mat(x)
        y = np.mat(y)
        self.w = (x.T * x).I * x.T * y
        return x * self.w
           #if __name__ == 'main':
for i in range(10, 51, 10):
    x = np.random.randint(1, 100, (40, i))
    y = np.random.randint(1, 100, (40, 1))
    reg_result = PolynomialRegression().fit(x, y).transform(x)
    import matplotlib.pyplot as plt
    fig = plt.figure(num = 1, figsize = (15, 8))
    plt.plot(np.arange(len(y)), y, label = 'real_y')
    plt.plot(np.arange(len(reg_result)), reg_result, label = 'pred_y')
    plt.legend()
    plt.show()

• 运行结果