1.5.最小二乘法求二元线性回归
最小二乘法求二元线性回归
y = 2 x 1 + ( − 3 ) ∗ x 2 + 4 y=2x_{1}+(-3)*x_{2}+4 y=2x1+(−3)∗x2+4
构造数据集
import numpy as np
np.random.seed(1)
X = np.random.normal(size=(1000,2),scale=1)#1000行2列，每个样本有一个特征
X
array([[ 1.62434536, -0.61175641],
[-0.52817175, -1.07296862],
[ 0.86540763, -2.3015387 ],
...,
[-0.44747327, 0.18858291],
[ 0.56091806, -0.92165905],
[ 0.64737512, 1.38682559]])
y=2*X[:,0]+(-3)*X[:,1]+4
import matplotlib.pyplot as plt
plt.scatter(X[:,0],y)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Cxw1RLeR-1588587707992)(output_5_1.png)]
plt.scatter(X[:,1],y)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-NYJwj9Ao-1588587707997)(output_6_1.png)]
from mpl_toolkits import mplot3d
ax = plt.axes(projection='3d')
ax.scatter3D(X[:,0],X[:,1],y)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BV32AVXq-1588587708002)(output_7_1.png)]
# 拼接数据
All_data = np.concatenate((X,y.reshape(1000,1)),axis = 1)
All_data
array([[ 1.62434536, -0.61175641, 9.08395997],
[-0.52817175, -1.07296862, 6.16256236],
[ 0.86540763, -2.3015387 , 12.63543135],
...,
[-0.44747327, 0.18858291, 2.53930472],
[ 0.56091806, -0.92165905, 7.88681328],
[ 0.64737512, 1.38682559, 1.13427345]])
# 拆分训练集和测试集
np.random.shuffle(All_data)
train_data = All_data[:700,:]
test_data = All_data[700:,:]
train_data.shape,test_data.shape
((700, 3), (300, 3))
构造模型
定义模型为 y = w 0 ∗ x 0 + w 1 ∗ x 1 + b y = w_{0}*x_{0}+w_{1}*x_{1}+b y=w0∗x0+w1∗x1+b
# 随机初始化模型参数
W = np.random.normal(size=(2))#normal正态分布
b = np.random.rand()
W[0],W[1],b
(-0.2842071018143058, -0.6775593663532857, 0.6597098038387995)
定义损失函数-平方损失函数
L o s s = 0.5 ∗ ∑ ( y − y h a t ) 2 Loss =0.5*\sum(y-y_{hat})^2 Loss=0.5∗∑(y−yhat)2
W[0]
-0.2842071018143058
loss = 0.5*np.sum((y-(2*X[:,0]+(-3)*X[:,1]+4))**2)
loss
0.0
# 可以看一下当前W，b下模型的损失值
loss = 0.5*np.sum((y-(W[0]*X[:,0]+W[1]*X[:,1]+b))**2)
loss
11242.981408753352
定义参数的优化算法并更新参数
梯度： − X ( y − X T w ) -X(y-X^{T}w) −X(y−XTw)
# 定义超参数
lr = 0.001 #学习率
X = train_data[:,:2]
X = X.reshape(2,700)
y = train_data[:,-1]
X.shape
(2, 700)
# 构造增广权重向量
W_hat = np.concatenate((W,np.array([b]))).reshape(3,1)
# 构造增广特征向量
X_hat = np.concatenate((X,np.ones((1,700))),axis=0)
W_hat.shape,X_hat.shape
((3, 1), (3, 700))
Num = 1#控制循环次数
w0_list = []
w1_list = []
b_list = []
loss_list = []
while True:
# 更新参数
W_hat = W_hat + lr * \
np.dot(X_hat,(y.reshape(700,1) - np.dot(X_hat.T,W_hat)))
# 计算经验错误
loss = np.sum((y.reshape(700,1) - np.dot(X_hat.T,W_hat))**2)/2
# 记录w，b和loss
w0_list.append(W_hat[0])
w1_list.append(W_hat[1])
b_list.append(W_hat[2])
loss_list.append(loss)
if Num % 10 == 1:
print("Num : %d , loss : %f"%(Num,loss))
Num = Num + 1
if loss < 1000 or Num > 100:
break
Num : 1 , loss : 5142.393184
Num : 11 , loss : 4763.128035
Num : 21 , loss : 4763.128035
Num : 31 , loss : 4763.128035
Num : 41 , loss : 4763.128035
Num : 51 , loss : 4763.128035
Num : 61 , loss : 4763.128035
Num : 71 , loss : 4763.128035
Num : 81 , loss : 4763.128035
Num : 91 , loss : 4763.128035
loss
4763.1280354125665
plt.plot(range(len(loss_list)),loss_list)
[]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-VxIsw1uA-1588587708011)(output_22_1.png)]
plt.plot(range(len(w0_list)),w0_list)
[]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-SnmKcuBr-1588587708017)(output_23_1.png)]
plt.plot(range(len(w1_list)),w1_list)
[]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-mUGZQt5M-1588587708021)(output_24_1.png)]
plt.plot(range(len(b_list)),b_list)
[]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-zLqLI7KU-1588587708024)(output_25_1.png)]
# 梯度下降法在二元方程上的使用