机器学习_感知机

    感知机是二分类的线性分类模型，输入为特征向量， 输出为类别(1,-1)。 感知机目的是求能够将输入的线性数据进行划分的超平面。对误分类样本求损失函数，使其最小从而得到超平面。

    感知机分为原始形式以及对偶形式。模型:

    sign（）为符号函数，小于零输出-1，大于等于0输出1

原始形式:

    (1) 选取初始w，b

    (2) 在训练集中选取数据(x，y)

    (3)如果y(w*x +b)<=0，更新权重（分类正确的时候不更新）

更新方法：

        

        

    (4) 转（2）直到没有误分类点，求出来的w，,b带入模型

---

对偶形式:

    (1)初始alpha=[0,0....] n个,n=样本个数,b=0学习率

    (2)计算gram矩阵https://zhuanlan.zhihu.com/p/105470826

    (3)如果 更新权重

    更新方法:

    

    

    (4)计算得到, b

    带入到x+b=0求出w

代码如下，如有错误请告知谢谢

import numpy as np


class percePtron:
    def __init__(self, data):
        self.weight = np.array([0.0, 0.0])
        self.bias = 0.0
        self.data = data

    def train(self, learning_rate = 1.):
        num_change = 0
        for epoch in range(10):
            print('训练第{0}次'.format(epoch+1))
            for input in data:
                pre = input[2]*(input[0]*self.weight[0] + input[1]*self.weight[1] + self.bias)
                if pre <= 0:
                    num_change += 1
                    print('误差为:', pre)

                    self.weight[0] = self.weight[0] + input[2] *input[0] *  learning_rate
                    self.weight[1] = self.weight[1] + input[2] *input[1] * learning_rate
                    self.bias = self.bias + input[2] * learning_rate
                    print('-------第{0}次更新权重'.format(num_change))
                    print("weight : ", self.weight)
                    print("bias : ", self.bias)


    def predict(self, input):
        out =input[0]*self.weight[0] + input[1]*self.weight[1] + self.bias
        if out >=0 :
            return 1
        else:
            return -1


class percePtron_Dual:
    def __init__(self, data):
        self.weight = np.array([0.0, 0.0])
        self.bias = 0.0
        self.data = data

    def train(self, learning_rate = 1.):
        train_data = self.data[:,:-1]
        train_label = self.data[:,-1:]
        m = train_data.shape[0]
        alpha = np.zeros(m)
        gram = np.zeros((m, m))
        for i in range(m):
            for j in range(m):
                gram[i][j] = train_data[i] * np.mat(train_data[j]).transpose()

        # print(gram)
        for i in range(100):
            for idx in range(m):
                tmp = 0
                for j in range(m):
                    tmp += alpha[j] * train_label[j] * gram[j, idx]
                tmp += self.bias
                if (train_label[idx] * tmp <= 0):
                    alpha[idx] = alpha[idx] + learning_rate
                    self.bias = self.bias + learning_rate * train_label[idx]
            # print(alpha, self.bias)

        for num in range(m):
            self.weight += alpha[num]*train_data[num]*train_label[num]


    def predict(self, input):
        out =self.weight[0]*input[0]+self.weight[1]*input[1] -3
        if out >=0 :
            return 1
        else:
            return -1


if __name__ == "__main__":
    data = np.array([[3, 3, 1], [4, 3, 1], [1, 1, -1]])

    per = percePtron(data)
    per.train()
    result_ = per.predict(input=[1, 0])
    print(result_)

    per_d = percePtron_Dual(data)
    per_d.train()
    result = per_d.predict(input=[1, 0])
    print(result)