【机器学习】'Stochastic' Gradient Descent随机梯度下降

随机梯度下降：指的是每下降一步，使用一条训练集来计算梯度值

# -*- coding:utf-8 -*-
# @Time:4/18/19 9:35 PM
# @Author:CIGA
# @Filename:03_stochastic_gradient_descent.py
# @Software:PyCharm
"""
随机梯度下降stochastic_gradient_descent
随机梯度下降：指的是每下降一步，使用一条训练集来计算梯度值
"""
import numpy as np
import random

X = 2 * np.random.rand(100, 1)
y = 4 + 3 * X + np.random.randn(100, 1)
X_b = np.c_[np.ones((100, 1)), X]
# print(X_b)

n_epochs = 500
a0 = 0.1
decay_rate = 1

m = 100
num = [i for i in range(100)]


def learning_schedule(epoch_num):
    return (1.0 / (decay_rate * epoch_num + 1)) * a0


theta = np.random.randn(2, 1)

# epoch 是轮次的意思,意思是用m个样本做一轮迭代
for epoch in range(n_epochs):
    # 生成100个不重复的随机数
    rand = random.sample(num, 100)
    for i in range(m):
        random_index = rand[i]
        xi = X_b[random_index:random_index + 1]
        yi = y[random_index:random_index + 1]
        gradients = xi.T.dot(xi.dot(theta) - yi)
        learning_rate = learning_schedule(epoch + i)
        theta = theta - learning_rate * gradients

print(theta)