最小二乘法-线性拟合

文章目录

    • 1、概述
    • 2、理论
    • 3、高斯消元法求解线性方程
      • 3.1、原则
      • 3.2、一阶线性函数

1、概述

最小二乘法是曲线拟合的常用方法,使用该方法对匹配函数的选取非常重要。

2、理论

  • 假设拟合的多项式为:

y = a 0 + a 1 ∗ x + a 2 ∗ x 2 + . . . + a m ∗ x m y = a_0 + a_1*x +a_2*x^2 + ...+a_m*x^m y=a0+a1x+a2x2+...+amxm
其中,m代表多项式的阶数。

  • 离散点与该多项式的平方和 F ( a 0 , a 1 , , a m ) F(a_0,a_1,,a_m) F(a0,a1,,am)为,其中n代表采样点数:
    F ( a 0 , a 1 , , a m ) = ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] 2 F(a_0,a_1,,a_m) = \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)]^2 F(a0,a1,,am)=i=0n[yi(a0+a1xi+a2xi2+...+amxim)]2
  • 最小二乘法的思想是求平方和函数 F ( a 0 , a 1 , , a m ) F(a_0,a_1,,a_m) F(a0,a1,,am)的最小值,而对于二次方程求最小值的问题,常见的思路就是对方程求导,倒数为零的点,及为方程的极值点。
  • 下面对 F ( a 0 , a 1 , , a m ) F(a_0,a_1,,a_m) F(a0,a1,,am)分别求 a i a_i ai偏导数,得:
    − 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] = 0 − 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] x i = 0 − 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] x i 2 = 0 … − 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] x i m = 0 -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] = 0 \\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i= 0\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^2= 0\\ \ldots\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^m= 0 2i=0n[yi(a0+a1xi+a2xi2+...+amxim)]=02i=0n[yi(a0+a1xi+a2xi2+...+amxim)]xi=02i=0n[yi(a0+a1xi+a2xi2+...+amxim)]xi2=02i=0n[yi(a0+a1xi+a2xi2+...+amxim)]xim=0
  • 整理得
    a 0 ∑ i = 0 n + a 1 ∑ i = 0 n x i + a 2 ∑ i = 0 n x i 2 + . . . + a m ∑ i = 0 n x i m = ∑ i = 0 n y i a 0 ∑ i = 0 n x i + a 1 ∑ i = 0 n x i 2 + a 2 ∑ i = 0 n x i 3 + . . . + a m ∑ i = 0 n x i m + 1 = ∑ i = 0 n y i x i … a 0 ∑ i = 0 n x i m + a 1 ∑ i = 0 n x i m + 1 + a 2 ∑ i = 0 n x i m + 2 + . . . + a m ∑ i = 0 n x i 2 m = ∑ i = 0 n y i x i m a_0\sum_{i=0}^{n}+a_1\sum_{i=0}^{n} x_i+ a_2\sum_{i=0}^{n} x_i^2+...+a_m\sum_{i=0}^{n}x_i^m = \sum_{i=0}^{n}y_i\\ a_0\sum_{i=0}^{n} x_i+a_1\sum_{i=0}^{n} x_i^2+ a_2\sum_{i=0}^{n} x_i^3+...+a_m\sum_{i=0}^{n}x_i^{m+1} = \sum_{i=0}^{n}y_ix_i\\ \ldots\\ a_0\sum_{i=0}^{n} x_i^m+a_1\sum_{i=0}^{n} x_i^{m+1}+ a_2\sum_{i=0}^{n} x_i^{m+2}+...+a_m\sum_{i=0}^{n}x_i^{2m} = \sum_{i=0}^{n}y_ix_i^m a0i=0n+a1i=0nxi+a2i=0nxi2+...+ami=0nxim=i=0nyia0i=0nxi+a1i=0nxi2+a2i=0nxi3+...+ami=0nxim+1=i=0nyixia0i=0nxim+a1i=0nxim+1+a2i=0nxim+2+...+ami=0nxi2m=i=0nyixim
  • 使用矩阵形式表示为:
    [ ∑ i = 0 n ∑ i = 0 n x i ∑ i = 0 n x i 2 … ∑ i = 0 n x i m ∑ i = 0 n x i ∑ i = 0 n x i 2 ∑ i = 0 n x i 3 … ∑ i = 0 n x i m + 1 ∑ i = 0 n x i 2 ∑ i = 0 n x i 3 ∑ i = 0 n x i 4 … ∑ i = 0 n x i m + 2 … … … … … ∑ i = 0 n x i m ∑ i = 0 n x i m + 1 ∑ i = 0 n x i m + 2 … ∑ i = 0 n x i 2 m ] [ a 0 a 1 a 2 … a m ] = [ ∑ i = 0 n y i ∑ i = 0 n y i x i ∑ i = 0 n y i x i 2 … ∑ i = 0 n y i x i m ] \left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\ldots&\sum_{i=0}^{n}x_i^m\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\ldots&\sum_{i=0}^{n}x_i^{m+1}\\ \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\sum_{i=0}^{n}x_i^4&\ldots&\sum_{i=0}^{n}x_i^{m+2}\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ \sum_{i=0}^{n}x_i^{m}&\sum_{i=0}^{n}x_i^{m+1}&\sum_{i=0}^{n}x_i^{m+2}&\ldots&\sum_{i=0}^{n}x_i^{2m} \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1\\ a_2\\ \ldots\\ a_m \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i\\ \sum_{i=0}^{n}y_ix_i^2\\ \ldots\\ \sum_{i=0}^{n}y_ix_i^m\\ \end{matrix} \right] i=0ni=0nxii=0nxi2i=0nximi=0nxii=0nxi2i=0nxi3i=0nxim+1i=0nxi2i=0nxi3i=0nxi4i=0nxim+2i=0nximi=0nxim+1i=0nxim+2i=0nxi2ma0a1a2am=i=0nyii=0nyixii=0nyixi2i=0nyixim
  • 下面就是求解一次线性方程,常用的方法是使用高斯消元法,也可以使用求矩阵的秩或者求解逆矩阵等方式求解。

3、高斯消元法求解线性方程

3.1、原则

  • 两个方程互换解不变
  • 一个方程乘以非零K,解不变
  • 一个方程乘以非零K,加上另一个方程解不变

3.2、一阶线性函数

假设多项式得阶数m为2,则上述矩阵方程为:
[ ∑ i = 0 n ∑ i = 0 n x i ∑ i = 0 n x i ∑ i = 0 n x i 2 ] [ a 0 a 1 ] = [ ∑ i = 0 n y i ∑ i = 0 n y i x i ] \left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2& \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1 \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i \end{matrix} \right] [i=0ni=0nxii=0nxii=0nxi2][a0a1]=[i=0nyii=0nyixi]
写成行列式的形式为:
∣ ∑ i = 0 n ∑ i = 0 n x i ∑ i = 0 n y i ∑ i = 0 n x i ∑ i = 0 n x i 2 ∑ i = 0 n y i x i ∣ \begin{vmatrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}x_i & \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}y_ix_i\\ \end{vmatrix} i=0ni=0nxii=0nxii=0nxi2i=0nyii=0nyixi
第一行乘以系数 ∑ i = 0 n \sum_{i=0}^{n} i=0n,第二行乘以系数 ∑ i = 0 n x i \sum_{i=0}^{n}x_i i=0nxi

∣ 1 ∑ i = 0 n x i ∑ i = 0 n ∑ i = 0 n y i ∑ i = 0 n 1 ∑ i = 0 n x i 2 ∑ i = 0 n x i ∑ i = 0 n y i x i ∑ i = 0 n x i ∣ \begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 1 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} \end{vmatrix} 11i=0ni=0nxii=0nxii=0nxi2i=0ni=0nyii=0nxii=0nyixi
第二行减去第一行得
∣ 1 ∑ i = 0 n x i ∑ i = 0 n ∑ i = 0 n y i ∑ i = 0 n 0 ∑ i = 0 n x i 2 ∑ i = 0 n x i − ∑ i = 0 n x i ∑ i = 0 n ∑ i = 0 n y i x i ∑ i = 0 n x i − ∑ i = 0 n y i ∑ i = 0 n ∣ \begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 0 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} \end{vmatrix} 10i=0ni=0nxii=0nxii=0nxi2i=0ni=0nxii=0ni=0nyii=0nxii=0nyixii=0ni=0nyi
通过上式可得 a 1 a_1 a1
a 1 = ∑ i = 0 n y i x i ∑ i = 0 n x i − ∑ i = 0 n y i ∑ i = 0 n ∑ i = 0 n x i 2 ∑ i = 0 n x i − ∑ i = 0 n x i ∑ i = 0 n = ∑ i = 0 n y i x i ∗ ∑ i = 0 n − ∑ i = 0 n y i ∗ ∑ i = 0 n x i ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i a_1 = \frac{\frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}}{\frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}}} =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} a1=i=0nxii=0nxi2i=0ni=0nxii=0nxii=0nyixii=0ni=0nyi=i=0nxi2i=0ni=0nxii=0nxii=0nyixii=0ni=0nyii=0nxi
同理将 a 1 a_1 a1回代入得:
a 0 = ∑ i = 0 n y i ∑ i = 0 n − ( ∑ i = 0 n y i x i ∗ ∑ i = 0 n − ∑ i = 0 n y i ∗ ∑ i = 0 n x i ) ∗ ∑ i = 0 n x i ( ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i ) ∗ ∑ i = 0 n = ( ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i ) ∗ ∑ i = 0 n y i − ( ∑ i = 0 n y i x i ∗ ∑ i = 0 n − ∑ i = 0 n y i ∗ ∑ i = 0 n x i ) ∗ ∑ i = 0 n x i ( ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i ) ∗ ∑ i = 0 n = ∑ i = 0 n x i 2 ∗ ∑ i = 0 n ∑ i = 0 n y i − ∑ i = 0 n y i x i ∗ ∑ i = 0 n ∑ i = 0 n x i ( ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i ) ∗ ∑ i = 0 n = ∑ i = 0 n x i 2 ∑ i = 0 n y i − ∑ i = 0 n y i x i ∑ i = 0 n x i ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i a_0 = \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} - \frac{(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}y_i -(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n}\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} a0=i=0ni=0nyi(i=0nxi2i=0ni=0nxii=0nxi)i=0n(i=0nyixii=0ni=0nyii=0nxi)i=0nxi=(i=0nxi2i=0ni=0nxii=0nxi)i=0n(i=0nxi2i=0ni=0nxii=0nxi)i=0nyi(i=0nyixii=0ni=0nyii=0nxi)i=0nxi=(i=0nxi2i=0ni=0nxii=0nxi)i=0ni=0nxi2i=0ni=0nyii=0nyixii=0ni=0nxi=i=0nxi2i=0ni=0nxii=0nxii=0nxi2i=0nyii=0nyixii=0nxi
故最终可得
a 0 = ∑ i = 0 n x i 2 ∑ i = 0 n y i − ∑ i = 0 n y i x i ∑ i = 0 n x i ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i a_0 =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} a0=i=0nxi2i=0ni=0nxii=0nxii=0nxi2i=0nyii=0nyixii=0nxi
a 1 = ∑ i = 0 n y i x i ∗ ∑ i = 0 n − ∑ i = 0 n y i ∗ ∑ i = 0 n x i ∑ i = 0 n x i 2 ∗ ∑ i = 0 n − ∑ i = 0 n x i ∗ ∑ i = 0 n x i a_1 =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} a1=i=0nxi2i=0ni=0nxii=0nxii=0nyixii=0ni=0nyii=0nxi

你可能感兴趣的:(基础算法)