机器学习（周志华）学习笔记五：第五章习题5.5

一、神经网络

神经网络通过隐层（中间层）的变换，增加了超平面的划分能力。然而，在对神经网络的学习中，没有看到关于隐层神经元数量选择的解释。也就是说，在神经网络理论中似乎并没有考虑到网络结构对输出结果产生的影响。

二、代码实现

    def calculate(self, input_values):
        """

        :param input_values: [color root ...]
        :return: [y1, y2 ...]
        """
        # 隐层神经元的输出
        bs = []
        for h in range(self.mid_num):
            alpha = 0
            for i in range(self.input_num):
                alpha += input_values[i] * self.neron_group['input'][i]['weight'][h]
            # 激活函数作用
            b = self.active_function(alpha - self.neron_group['mid'][h]['threshold'])
            bs.append(b)

        # 输出层神经元的输出
        ys = []
        for j in range(self.output_num):
            beta = 0
            for h in range(self.mid_num):
                beta += bs[h]*self.neron_group['mid'][h]['weight'][j]
            # 激活作用
            y = self.active_function(beta - self.neron_group['output'][j]['threshold'])
            ys.append(y)

        return ys, bs

    def learn(self, d_samples, learning_rate):
        """
        学习。此神经网络输出是一维的
        :param learning_rate: 学习率
        :param d_samples: 存储数据集样例的标号
        :return:
        """
        self.create_neuron_group(self.input_num, self.mid_num, self.output_num)
        previous_e = -1
        now_e = 0
        count = 0
        while True:

            for sample in d_samples:
                input_values = []
                for index in self.attrs_index:
                    # tag为样例在某一特定属性上的取值
                    tag = self.attrs_list[index][sample]
                    if type(tag) == str:
                        tag = self.numerical(self.attrs_name[index], tag)
                    input_values.append(tag)
                yk, bs = self.calculate(input_values)

                gs = []
                for j in range(self.output_num):
                    gj = yk[j]*(1-yk[j])*(self.y_list[sample]-yk[j])
                    gs.append(gj)
                    now_e += 0.5 * pow((yk[j]-self.y_list[sample]), 2)

                es = []
                for h in range(self.mid_num):
                    eh = 0
                    for j in range(self.output_num):
                        eh += bs[h] * (1 - bs[h]) * gs[j] * self.neron_group['mid'][h]['weight'][j]
                    es.append(eh)

                # 四个更新公式
                for h in range(self.mid_num):
                    for j in range(self.output_num):
                        delta_whj = learning_rate * gs[j] * bs[h]
                        self.neron_group['mid'][h]['weight'][j] += delta_whj

                for j in range(self.output_num):
                    delta_theta_j = -learning_rate * gs[j]
                    self.neron_group['output'][j]['threshold'] += delta_theta_j

                # input_num与attrs_index的关系需要完善
                for i in range(self.input_num):
                    for h in range(self.output_num):
                        delta_vih = learning_rate * es[h] * input_values[i]
                        self.neron_group['input'][i]['weight'][h] += delta_vih

                for h in range(self.mid_num):
                    delta_rh = -learning_rate * es[h]
                    self.neron_group['mid'][h]['threshold'] += delta_rh

            count += 1

            # 停止条件
            # if abs(now_e-previous_e) < 10**-6 or count > 2000:
            if abs(now_e - previous_e) < 0.0001:
                break
            else:
                previous_e = now_e
                now_e = 0

三、补充

这个程序是在很久以前写下的，只记得程序的功能，关于学习正确率的测试已经忘记了。程序源码：【BP算法神经网络.zip】