Fridaynoy

【TensorFlow自学2】基础语句+激活函数+损失函数+正则化缓解过拟合+优化器（SGD、Adam等）

2.1预备知识

tf.where()–条件语句真的话返回A，假的话返回B

tf.where(条件语句,A(真-返回),B(假-返回))

import tensorflow as tf

a=tf.constant([1,2,3,1,1])
b=tf.constant([0,1,3,4,5])
c=tf.where(tf.greater(a,b),a,b) #判断是否a>b，如果是返回a的值，不是返回b（输出两个数组中较大的元素）
print("c: ",c)

c:  tf.Tensor([1 2 3 4 5], shape=(5,), dtype=int32)

np.random.RandomState.rand()–返回一个[0,1)之间的随机数

np.random.RandomState.rand(维度) #维度为空时返回标量

import numpy as np

rdm=np.random.RandomState(seed=1) #seed=常数 每次生成的随机数相同
a=rdm.rand() #返回一个标量
b=rdm.rand(2,3) #返回一个2行3列的随机矩阵
print("a:",a)
print("b:",b)

a: 0.417022004702574
b: [[7.20324493e-01 1.14374817e-04 3.02332573e-01]
 [1.46755891e-01 9.23385948e-02 1.86260211e-01]]

np.vstack()–将两个数组按垂直方向叠加(堆叠)

np.vstack(数组1,数组2)

import numpy as np

a=np.array([1,2,3])
b=np.array([4,5,6])
c=np.stack((a,b))
print("c:\n",c)

c:
 [[1 2 3]
 [4 5 6]]

np.mgrid np.ravel() np.c_[]–生成网格

np.mgrid[起始值:结束值:步长,起始值:结束值:步长,起始值:结束值:步长,....]
x.ravel()--将x变成一维数组（把矩阵拉直）
np.c_[数组1,数组2,...]--使返回的间隔数值点配对

import numpy as np

x,y=np.mgrid[1:3:1,2:4:0.5]
grid=np.c_[x.ravel(),y.ravel()]
print("x:",x)
print("y:",y)
print("grid:\n",grid)

x: [[1. 1. 1. 1.]
 [2. 2. 2. 2.]]
y: [[2.  2.5 3.  3.5]
 [2.  2.5 3.  3.5]]
grid:
 [[1.  2. ]
 [1.  2.5]
 [1.  3. ]
 [1.  3.5]
 [2.  2. ]
 [2.  2.5]
 [2.  3. ]
 [2.  3.5]]

2.2神经网络(NN)复杂度

NN复杂度–多用NN层数和NN参数个数来表示

空间复杂度：

层数=隐藏层的层数+1个输出层（不算输入层）

总参数=总W+总b

时间复杂度：

乘加运算次数

学习率的选取：过大无法找到最优、过小迭代次数较大才能找到

指数衰减学习率：先用较大的学习率，快速找到最优解，然后逐步减小学习率，使得模型在训练后期稳定

指数衰减学习率=初始学习率*学习率衰减率^{当前轮数/多少轮衰减一次}

epoch=40
LR_BASE=0.2
LR_DECAY=0.99
LR_STEP=1

lr=LR_BASE*LR_DECAY**(epoch/LR_STEP) #添加进循环

import tensorflow as tf

w=tf.Variable(tf.constant(5,dtype=tf.float32))
epoch=40
LR_BASE=0.2
LR_DECAY=0.99
LR_STEP=1

for epoch in range(epoch): #对数据集的循环次数
    lr=LR_BASE*LR_DECAY**(epoch/LR_STEP) #添加进循环
    with tf.GradientTape() as tape: # with...grads是loss函数对w求梯度的过程
        loss=tf.square(w+1) #损失函数为$L=(w+1)^2$
    grads=tape.gradient(loss,w) # .gradient函数用来说明A对B求导

    w.assign_sub(lr*grads) #参数自己减梯度（向负梯度方向前进）
    print("After %s epoch, w is %f, loss is %f" %(epoch,w.numpy(),loss))

# 有效的找到最优解

After 0 epoch, w is 2.600000, loss is 36.000000
After 1 epoch, w is 1.174400, loss is 12.959999
After 2 epoch, w is 0.321948, loss is 4.728015
After 3 epoch, w is -0.191126, loss is 1.747547
After 4 epoch, w is -0.501926, loss is 0.654277
After 5 epoch, w is -0.691392, loss is 0.248077
After 6 epoch, w is -0.807611, loss is 0.095239
After 7 epoch, w is -0.879339, loss is 0.037014
After 8 epoch, w is -0.923874, loss is 0.014559
After 9 epoch, w is -0.951691, loss is 0.005795
After 10 epoch, w is -0.969167, loss is 0.002334
After 11 epoch, w is -0.980209, loss is 0.000951
After 12 epoch, w is -0.987226, loss is 0.000392
After 13 epoch, w is -0.991710, loss is 0.000163
After 14 epoch, w is -0.994591, loss is 0.000069
After 15 epoch, w is -0.996452, loss is 0.000029
After 16 epoch, w is -0.997660, loss is 0.000013
After 17 epoch, w is -0.998449, loss is 0.000005
After 18 epoch, w is -0.998967, loss is 0.000002
After 19 epoch, w is -0.999308, loss is 0.000001
After 20 epoch, w is -0.999535, loss is 0.000000
After 21 epoch, w is -0.999685, loss is 0.000000
After 22 epoch, w is -0.999786, loss is 0.000000
After 23 epoch, w is -0.999854, loss is 0.000000
After 24 epoch, w is -0.999900, loss is 0.000000
After 25 epoch, w is -0.999931, loss is 0.000000
After 26 epoch, w is -0.999952, loss is 0.000000
After 27 epoch, w is -0.999967, loss is 0.000000
After 28 epoch, w is -0.999977, loss is 0.000000
After 29 epoch, w is -0.999984, loss is 0.000000
After 30 epoch, w is -0.999989, loss is 0.000000
After 31 epoch, w is -0.999992, loss is 0.000000
After 32 epoch, w is -0.999994, loss is 0.000000
After 33 epoch, w is -0.999996, loss is 0.000000
After 34 epoch, w is -0.999997, loss is 0.000000
After 35 epoch, w is -0.999998, loss is 0.000000
After 36 epoch, w is -0.999999, loss is 0.000000
After 37 epoch, w is -0.999999, loss is 0.000000
After 38 epoch, w is -0.999999, loss is 0.000000
After 39 epoch, w is -0.999999, loss is 0.000000

2.3激活函数

1)Sigmoid函数–tf.nn.sigmoid(x)
$f(x)=\frac{1}{1+e^{-x}}$
易造成梯度消失（多个0-0.25的数字相乘）、收敛慢、幂运算复杂训练时间长

2)Tanh函数–tf.math.tanh(x)
$f(x)=\frac{1-e^{-2x}}{1+e^{-2x}}$
输出是0均值、梯度消失、幂运算复杂

3)ReLU函数–tf.nn.relu(x)
$f(x)=\max(x,0)$
解决了梯度消失问题、计算简单速度快、收敛快
输出非0均值，收敛慢、DeadReLU问题（某些神经元可能永远不被激活，对应参数无法更新）

4)Leaky_ReLU函数–tf.nn.leaky_relu(x)
$f(x)=\max(\alpha x,x)$
解决了DeadReLU问题，但实际操作中并不是完全优于ReLU

✨Tips:

首选ReLU函数、学习率设置较小值、输入特征标准化（标准正态分布）、初始参数中心化（随机生成的初始参数满足0均值 $\sqrt{2/feature\ number}$ 标准差的正态分布）

2.4损失函数（loss）

loss：预测值（y）与已知真实值（y_）的差距

NN的优化目标：loss最小–MSE(Mean Squared Error)、ce(Cross Entropy)、自定义

(1)均方误差MSE–loss_mse=tf.reduce_mean(tf.square(y_-y))
$MSE(y\_,y)=\frac{1}{n}\sum_{i=1}^n(y\_-y)^2$

#预测酸奶日销量y
#假设x1与x2是影响销量的因素，自建一组数据：随机生成x1和x2，计算x1+x2，并添加噪声：-0.05～0.05，作为观测真实值y_

import tensorflow as tf 
import numpy as np

SEED=23455

rdm=np.random.RandomState(seed=SEED) #生成[0,1)之间的随机数
x=rdm.rand(32,2) #32行2列的矩阵，32组真实数据
y_=[[x1+x2+(rdm.rand()/10.0-0.05)] for (x1,x2) in x] #生成带噪声的真实值
x=tf.cast(x,dtype=tf.float32)

w1=tf.Variable(tf.random.normal([2,1],stddev=1,seed=1))

epoch=15000
lr=0.002

for epoch in range(epoch):
    with tf.GradientTape() as tape:
        y=tf.matmul(x,w1)
        loss_mse=tf.reduce_mean(tf.square(y_-y))
    
    grads=tape.gradient(loss_mse,w1)
    w1.assign_sub(lr*grads)

    if epoch%500==0:
        print("After %d training steps, w1 is " %(epoch))
        print(w1.numpy(),"\n")
print("final w1 is: ", w1.numpy())

After 0 training steps, w1 is 
[[ 0.8266879 ]
 [-0.68062395]] 

After 500 training steps, w1 is 
[[1.2981918 ]
 [0.12153205]] 

After 1000 training steps, w1 is 
[[1.3930992]
 [0.4468745]] 

After 1500 training steps, w1 is 
[[1.3728648 ]
 [0.60195106]] 

After 2000 training steps, w1 is 
[[1.3233058]
 [0.691584 ]] 

After 2500 training steps, w1 is 
[[1.2715786]
 [0.7524795]] 

After 3000 training steps, w1 is 
[[1.2254242]
 [0.7981871]] 

After 3500 training steps, w1 is 
[[1.1863139 ]
 [0.83425367]] 

After 4000 training steps, w1 is 
[[1.153813  ]
 [0.86335814]] 

After 4500 training steps, w1 is 
[[1.1270124 ]
 [0.88707036]] 

After 5000 training steps, w1 is 
[[1.1049817]
 [0.9064663]] 

After 5500 training steps, w1 is 
[[1.0868948 ]
 [0.92235756]] 

After 6000 training steps, w1 is 
[[1.0720533]
 [0.9353866]] 

After 6500 training steps, w1 is 
[[1.0598779 ]
 [0.94607174]] 

After 7000 training steps, w1 is 
[[1.0498903]
 [0.9548356]] 

After 7500 training steps, w1 is 
[[1.0416976 ]
 [0.96202403]] 

After 8000 training steps, w1 is 
[[1.0349776]
 [0.9679204]] 

After 8500 training steps, w1 is 
[[1.0294647]
 [0.9727569]] 

After 9000 training steps, w1 is 
[[1.0249432 ]
 [0.97672427]] 

After 9500 training steps, w1 is 
[[1.0212348]
 [0.9799782]] 

After 10000 training steps, w1 is 
[[1.0181928 ]
 [0.98264736]] 

After 10500 training steps, w1 is 
[[1.0156975 ]
 [0.98483694]] 

After 11000 training steps, w1 is 
[[1.0136509]
 [0.9866328]] 

After 11500 training steps, w1 is 
[[1.0119709]
 [0.9881056]] 

After 12000 training steps, w1 is 
[[1.0105952 ]
 [0.98931384]] 

After 12500 training steps, w1 is 
[[1.009466  ]
 [0.99030447]] 

After 13000 training steps, w1 is 
[[1.0085402]
 [0.9911175]] 

After 13500 training steps, w1 is 
[[1.0077778 ]
 [0.99178374]] 

After 14000 training steps, w1 is 
[[1.007155 ]
 [0.9923309]] 

After 14500 training steps, w1 is 
[[1.0066465]
 [0.9927791]] 

final w1 is:  [[1.0062273]
 [0.9931454]]

(2)自定义损失函数

上述例子中，如果酸奶的成本和利润是一定的，那么预测过多损失的是成本；预测少损失的是利润。

当成本与利润不等时，例如酸奶的成本是1元，售价为100元，那么利润就是99元，我们更加希望预测值不完全准确时倾向于往多了预测。

因此我们自定义损失函数：
$KaTeX parse error: No such environment: equation* at position 8: \begin{̲e̲q̲u̲a̲t̲i̲o̲n̲*̲}̲ f(y\_,y)= \lef…$

import tensorflow as tf 
import numpy as np

SEED=23455

rdm=np.random.RandomState(seed=SEED) #生成[0,1)之间的随机数
x=rdm.rand(32,2) #32行2列的矩阵，32组真实数据
y_=[[x1+x2+(rdm.rand()/10.0-0.05)] for (x1,x2) in x] #生成带噪声的真实值
x=tf.cast(x,dtype=tf.float32)

w1=tf.Variable(tf.random.normal([2,1],stddev=1,seed=1))

epoch=15000
lr=0.002
cost=1
profit=99

for epoch in range(epoch):
    with tf.GradientTape() as tape:
        y=tf.matmul(x,w1)
        loss=tf.reduce_sum(tf.where(tf.greater(y,y_),(y-y_)*cost,(y_-y)*profit)) #只改变了损失函数
    
    grads=tape.gradient(loss,w1)
    w1.assign_sub(lr*grads)

    if epoch%500==0:
        print("After %d training steps, w1 is " %(epoch))
        print(w1.numpy(),"\n")
print("final w1 is: ", w1.numpy())

After 0 training steps, w1 is 
[[1.8674549]
 [3.9414268]] 

After 500 training steps, w1 is 
[[1.156094 ]
 [1.0404317]] 

After 1000 training steps, w1 is 
[[1.1464739]
 [1.0717155]] 

After 1500 training steps, w1 is 
[[1.136854 ]
 [1.1029994]] 

After 2000 training steps, w1 is 
[[1.1272339]
 [1.1342831]] 

After 2500 training steps, w1 is 
[[1.1762469]
 [1.1768597]] 

After 3000 training steps, w1 is 
[[1.1458087]
 [1.0316733]] 

After 3500 training steps, w1 is 
[[1.1361889]
 [1.0629573]] 

After 4000 training steps, w1 is 
[[1.1265689]
 [1.0942411]] 

After 4500 training steps, w1 is 
[[1.1755823]
 [1.1368182]] 

After 5000 training steps, w1 is 
[[1.1659625]
 [1.1681021]] 

After 5500 training steps, w1 is 
[[1.135524 ]
 [1.0229155]] 

After 6000 training steps, w1 is 
[[1.125904 ]
 [1.0541992]] 

After 6500 training steps, w1 is 
[[1.1162838]
 [1.0854828]] 

After 7000 training steps, w1 is 
[[1.1066637]
 [1.1167666]] 

After 7500 training steps, w1 is 
[[1.1556768]
 [1.1593434]] 

After 8000 training steps, w1 is 
[[1.1252388]
 [1.014157 ]] 

After 8500 training steps, w1 is 
[[1.1156188]
 [1.0454409]] 

After 9000 training steps, w1 is 
[[1.1059989]
 [1.0767248]] 

After 9500 training steps, w1 is 
[[1.1550124]
 [1.1193019]] 

After 10000 training steps, w1 is 
[[1.1453927]
 [1.150586 ]] 

After 10500 training steps, w1 is 
[[1.1944054]
 [1.1931624]] 

After 11000 training steps, w1 is 
[[1.1639671]
 [1.0479759]] 

After 11500 training steps, w1 is 
[[1.1543471]
 [1.0792596]] 

After 12000 training steps, w1 is 
[[1.1447271]
 [1.1105435]] 

After 12500 training steps, w1 is 
[[1.135107 ]
 [1.1418272]] 

After 13000 training steps, w1 is 
[[1.1841207]
 [1.1844045]] 

After 13500 training steps, w1 is 
[[1.1536822]
 [1.0392178]] 

After 14000 training steps, w1 is 
[[1.1440623]
 [1.0705017]] 

After 14500 training steps, w1 is 
[[1.1344426]
 [1.1017858]] 

final w1 is:  [[1.1359003]
 [1.155832 ]]

(3)交叉熵函数CE(Cross Entropy)表示两个概率分布之间的距离
$H(y\_,y)=-\sum y\_*\ln y$
值越小，概率分布越接近–tf.losses.categorical_crossentropy(y_,y)

softmax与交叉熵结合：输出先过softmax函数，再计算y与y_的交叉熵损失函数

–tf.nn.softmax_cross_entropy_with_losits(y_,y)

2.5缓解过拟合

欠拟合：学习的不够彻底，使模型不能有效拟合数据集–【增加特征、增加参数、减少正则化参数】

过拟合：模型对当前数据拟合的过好，缺乏泛化力（对没见过的数据预测不好）–【数据清洗减少噪声、增大训练集、采用正则化、增大正则化参数】

✅正则化缓解过拟合

正则化：在损失函数中引入模型复杂度，利用给参数阵W加权值，弱化训练的噪声（一般情况不会正则化b）
$loss=loss(y,y\_)+regularizer*loss(w)$

$loss_{L1}(w)=\sum_i |w_i|\qquad loss_{L2}(w)=\sum_i |w_i^2|$

L1正则化：会使很多参数变为0，该方法通过稀疏参数（减少参数数量）来降低复杂度

L2正则化：会使参数接近0，但是不为0，通过减小值的大小来缓解数据噪声引起的过拟合

#不带正则化的情形

# 导入所需模块
import tensorflow as tf
from matplotlib import pyplot as plt
import numpy as np
import pandas as pd

# 读入数据/标签 生成x_train y_train
df = pd.read_csv('/Users/viva/Downloads/class2/dot.csv')
x_data = np.array(df[['x1', 'x2']])
y_data = np.array(df['y_c'])

x_train = np.vstack(x_data).reshape(-1,2) #vstack将数据按行堆叠起来
y_train = np.vstack(y_data).reshape(-1,1) #reshape中-1代表自动计算，2表示确定两列，行数自动计算

Y_c = [['red' if y else 'blue'] for y in y_train]

# 转换x的数据类型，否则后面矩阵相乘时会因数据类型问题报错
x_train = tf.cast(x_train, tf.float32)
y_train = tf.cast(y_train, tf.float32)

# from_tensor_slices函数切分传入的张量的第一个维度，生成相应的数据集，使输入特征和标签值一一对应
train_db = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(32)

# 生成神经网络的参数，输入层为2个神经元，隐藏层为11个神经元，1层隐藏层，输出层为1个神经元
# 用tf.Variable()保证参数可训练
w1 = tf.Variable(tf.random.normal([2, 11]), dtype=tf.float32)
b1 = tf.Variable(tf.constant(0.01, shape=[11])) #这里的11与w1的列数相同，保持同样的维度

w2 = tf.Variable(tf.random.normal([11, 1]), dtype=tf.float32) #第二层的输入11要与第一层输出数量相同
b2 = tf.Variable(tf.constant(0.01, shape=[1]))

lr = 0.01  # 学习率
epoch = 400  # 循环轮数

# 训练部分
for epoch in range(epoch):
    for step, (x_train, y_train) in enumerate(train_db):
        with tf.GradientTape() as tape:  # 记录梯度信息

            h1 = tf.matmul(x_train, w1) + b1  # 记录神经网络乘加运算
            h1 = tf.nn.relu(h1) #带有激活函数
            y = tf.matmul(h1, w2) + b2

            # 采用均方误差损失函数mse = mean(sum(y-out)^2)
            loss = tf.reduce_mean(tf.square(y_train - y))

        # 计算loss对各个参数的梯度
        variables = [w1, b1, w2, b2]
        grads = tape.gradient(loss, variables)

        # 实现梯度更新
        # w1 = w1 - lr * w1_grad tape.gradient是自动求导结果与[w1, b1, w2, b2] 索引为0，1，2，3 
        w1.assign_sub(lr * grads[0])
        b1.assign_sub(lr * grads[1])
        w2.assign_sub(lr * grads[2])
        b2.assign_sub(lr * grads[3])

    # 每20个epoch，打印loss信息
    if epoch % 20 == 0:
        print('epoch:', epoch, 'loss:', float(loss))

# 预测部分
print("*******predict*******")
# xx在-3到3之间以步长为0.1，yy在-3到3之间以步长0.01,生成间隔数值点
xx, yy = np.mgrid[-3:3:.1, -3:3:.1]
# 将xx , yy拉直，并合并配对为二维张量，生成二维坐标点
grid = np.c_[xx.ravel(), yy.ravel()]
grid = tf.cast(grid, tf.float32)
# 将网格坐标点喂入神经网络，进行预测，probs为输出
probs = []
for x_test in grid:
    # 使用训练好的参数进行预测
    h1 = tf.matmul([x_test], w1) + b1
    h1 = tf.nn.relu(h1)
    y = tf.matmul(h1, w2) + b2  # y为预测结果
    probs.append(y)

# 取第0列给x1，取第1列给x2
x1 = x_data[:, 0]
x2 = x_data[:, 1]
# probs的shape调整成xx的样子
probs = np.array(probs).reshape(xx.shape)
plt.scatter(x1, x2, color=np.squeeze(Y_c)) #squeeze去掉纬度是1的纬度,相当于去掉[['red'],[''blue]],内层括号变为['red','blue']
# 把坐标xx yy和对应的值probs放入contour<[‘kɑntʊr]>函数，给probs值为0.5的所有点上色  plt点show后 显示的是红蓝点的分界线
plt.contour(xx, yy, probs, levels=[.5])
plt.show()

# 读入红蓝点，画出分割线，不包含正则化
# 不清楚的数据，建议print出来查看

epoch: 0 loss: 0.6055795550346375
epoch: 20 loss: 0.07649826258420944
epoch: 40 loss: 0.06883294880390167
epoch: 60 loss: 0.06506730616092682
epoch: 80 loss: 0.060232751071453094
epoch: 100 loss: 0.056612174957990646
epoch: 120 loss: 0.05371078848838806
epoch: 140 loss: 0.052048470824956894
epoch: 160 loss: 0.05126640945672989
epoch: 180 loss: 0.05126921460032463
epoch: 200 loss: 0.05139847844839096
epoch: 220 loss: 0.04914329573512077
epoch: 240 loss: 0.046171944588422775
epoch: 260 loss: 0.04357674717903137
epoch: 280 loss: 0.0416145883500576
epoch: 300 loss: 0.03968871012330055
epoch: 320 loss: 0.037830088287591934
epoch: 340 loss: 0.036351241171360016
epoch: 360 loss: 0.03420327603816986
epoch: 380 loss: 0.03207312151789665
*******predict*******

#带有L2正则化的情形

# 导入所需模块
import tensorflow as tf
from matplotlib import pyplot as plt
import numpy as np
import pandas as pd

# 读入数据/标签 生成x_train y_train
df = pd.read_csv('/Users/viva/Downloads/class2/dot.csv')
x_data = np.array(df[['x1', 'x2']])
y_data = np.array(df['y_c'])

x_train = x_data
y_train = y_data.reshape(-1, 1)

Y_c = [['red' if y else 'blue'] for y in y_train]

# 转换x的数据类型，否则后面矩阵相乘时会因数据类型问题报错
x_train = tf.cast(x_train, tf.float32)
y_train = tf.cast(y_train, tf.float32)

# from_tensor_slices函数切分传入的张量的第一个维度，生成相应的数据集，使输入特征和标签值一一对应
train_db = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(32)

# 生成神经网络的参数，输入层为4个神经元，隐藏层为32个神经元，2层隐藏层，输出层为3个神经元
# 用tf.Variable()保证参数可训练
w1 = tf.Variable(tf.random.normal([2, 11]), dtype=tf.float32)
b1 = tf.Variable(tf.constant(0.01, shape=[11]))

w2 = tf.Variable(tf.random.normal([11, 1]), dtype=tf.float32)
b2 = tf.Variable(tf.constant(0.01, shape=[1]))

lr = 0.01  # 学习率为
epoch = 400  # 循环轮数

# 训练部分
for epoch in range(epoch):
    for step, (x_train, y_train) in enumerate(train_db):
        with tf.GradientTape() as tape:  # 记录梯度信息

            h1 = tf.matmul(x_train, w1) + b1  # 记录神经网络乘加运算
            h1 = tf.nn.relu(h1)
            y = tf.matmul(h1, w2) + b2

            # 采用均方误差损失函数mse = mean(sum(y-out)^2)
            loss_mse = tf.reduce_mean(tf.square(y_train - y))
            # 添加l2正则化
            loss_regularization = []
            # tf.nn.l2_loss(w)=sum(w ** 2) / 2
            loss_regularization.append(tf.nn.l2_loss(w1))
            loss_regularization.append(tf.nn.l2_loss(w2))
            # 求和
            # 例：x=tf.constant(([1,1,1],[1,1,1]))
            #   tf.reduce_sum(x)
            # >>>6
            # loss_regularization = tf.reduce_sum(tf.stack(loss_regularization))
            loss_regularization = tf.reduce_sum(loss_regularization)
            loss = loss_mse + 0.03 * loss_regularization #REGULARIZER = 0.03

        # 计算loss对各个参数的梯度
        variables = [w1, b1, w2, b2]
        grads = tape.gradient(loss, variables)

        # 实现梯度更新
        # w1 = w1 - lr * w1_grad
        w1.assign_sub(lr * grads[0])
        b1.assign_sub(lr * grads[1])
        w2.assign_sub(lr * grads[2])
        b2.assign_sub(lr * grads[3])

    # 每200个epoch，打印loss信息
    if epoch % 20 == 0:
        print('epoch:', epoch, 'loss:', float(loss))

# 预测部分
print("*******predict*******")
# xx在-3到3之间以步长为0.01，yy在-3到3之间以步长0.01,生成间隔数值点
xx, yy = np.mgrid[-3:3:.1, -3:3:.1]
# 将xx, yy拉直，并合并配对为二维张量，生成二维坐标点
grid = np.c_[xx.ravel(), yy.ravel()]
grid = tf.cast(grid, tf.float32)
# 将网格坐标点喂入神经网络，进行预测，probs为输出
probs = []
for x_predict in grid:
    # 使用训练好的参数进行预测
    h1 = tf.matmul([x_predict], w1) + b1
    h1 = tf.nn.relu(h1)
    y = tf.matmul(h1, w2) + b2  # y为预测结果
    probs.append(y)

# 取第0列给x1，取第1列给x2
x1 = x_data[:, 0]
x2 = x_data[:, 1]
# probs的shape调整成xx的样子
probs = np.array(probs).reshape(xx.shape)
plt.scatter(x1, x2, color=np.squeeze(Y_c))
# 把坐标xx yy和对应的值probs放入contour<[‘kɑntʊr]>函数，给probs值为0.5的所有点上色  plt点show后 显示的是红蓝点的分界线
plt.contour(xx, yy, probs, levels=[.5])
plt.show()

# 读入红蓝点，画出分割线，包含正则化
# 不清楚的数据，建议print出来查看

epoch: 0 loss: 0.6997354030609131
epoch: 20 loss: 0.39997491240501404
epoch: 40 loss: 0.34798115491867065
epoch: 60 loss: 0.3107355535030365
epoch: 80 loss: 0.2789038121700287
epoch: 100 loss: 0.24940520524978638
epoch: 120 loss: 0.22390803694725037
epoch: 140 loss: 0.20139609277248383
epoch: 160 loss: 0.18274584412574768
epoch: 180 loss: 0.16758093237876892
epoch: 200 loss: 0.15457896888256073
epoch: 220 loss: 0.14414116740226746
epoch: 240 loss: 0.1351311355829239
epoch: 260 loss: 0.12741810083389282
epoch: 280 loss: 0.12007804214954376
epoch: 300 loss: 0.11288371682167053
epoch: 320 loss: 0.10653729736804962
epoch: 340 loss: 0.10097507387399673
epoch: 360 loss: 0.0966401919722557
epoch: 380 loss: 0.09320375323295593
*******predict*******

2.6神经网络参数优化器

待优化的参数为w、损失函数loss、学习率lr、每次迭代一个batch、t表示当前batch迭代的总次数

优化步骤：

1.计算t时刻损失函数关于当前参数的梯度： $g_t=\nabla loss$

2.计算t时刻的一阶动量 $m_t$ （与梯度相关的函数）和二阶动量 $V_t$ （与梯度平方相关的函数）

3.计算t时刻下降梯度： $\eta_t=lr\cdot m_t/\sqrt V_t$

4.计算t+1时刻的参数： $w_{t+1}=w_t-\eta_t$

优化器种类：

$\cdot$ SGD（无momentum动量）常用的梯度下降法

– $m_t=g_t, \ V_t=1$
$w_{t+1}=w_t-lr* \frac{\partial loss}{\partial w_t}$

$\cdot$ SGDM（含有momentum动量）SGD增加一阶动量

– $m_t=\beta \cdot m_{t-1}+(1-\beta)\cdot g_t, \ V_t=1$ （ $\beta$ 经验取值0.9）
$w_{t+1}=w_t-lr*(\beta \cdot m_{t}+(1-\beta)\cdot g_t)$

$\cdot$ Adagrad 在SGD的基础上增加二阶动量（对模型中的参数分配自适应学习率）

– $m_t=g_t, \ V_t=\sum_{\tau=1}^t g_{\tau}^2$
$w_{t+1}=w_t-lr*g_t/(\sqrt{\sum_{\tau=1}^t g_{\tau}^2})$

$\cdot$ RMSProp 在SGD的基础上增加二阶动量，使用指数滑动平均值计算，表示过去时间的平均值

– $m_t=g_t, \ V_t=\beta\cdot V_{t-1}+(1-\beta)\cdot g_t^2$
$w_{t+1}=w_t-lr*g_t/(\sqrt{\beta\cdot V_{t}+(1-\beta)\cdot g_t^2})$

$\cdot$ Adam 结合SGDM的一阶动量与RMSProp的二阶动量

– $m_t=\beta_1 \cdot m_{t-1}+(1-\beta_1)\cdot g_t, \ V_t=\beta_2 \cdot V_{t-1}+(1-\beta_2)\cdot g_t^2$
$w_{t+1}=w_t-lr\cdot \frac{m_t}{1-\beta_1^t}/\sqrt{\frac{V_t}{1-\beta_2^t}}$

# 利用鸢尾花数据集，实现前向传播、反向传播，可视化loss曲线--Adam优化器

# 导入所需模块
import tensorflow as tf
from sklearn import datasets
import matplotlib.pyplot as plt
import numpy as np
import time  ##1##

# 导入数据，分别为输入特征和标签
x_data = datasets.load_iris().data
y_data = datasets.load_iris().target

# 随机打乱数据（因为原始数据是顺序的，顺序不打乱会影响准确率）
# seed: 随机数种子，是一个整数，当设置之后，每次生成的随机数都一样（为方便教学，以保每位同学结果一致）
np.random.seed(116)  # 使用相同的seed，保证输入特征和标签一一对应
np.random.shuffle(x_data)
np.random.seed(116)
np.random.shuffle(y_data)
tf.random.set_seed(116)

# 将打乱后的数据集分割为训练集和测试集，训练集为前120行，测试集为后30行
x_train = x_data[:-30]
y_train = y_data[:-30]
x_test = x_data[-30:]
y_test = y_data[-30:]

# 转换x的数据类型，否则后面矩阵相乘时会因数据类型不一致报错
x_train = tf.cast(x_train, tf.float32)
x_test = tf.cast(x_test, tf.float32)

# from_tensor_slices函数使输入特征和标签值一一对应。（把数据集分批次，每个批次batch组数据）
train_db = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(32)
test_db = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)

# 生成神经网络的参数，4个输入特征故，输入层为4个输入节点；因为3分类，故输出层为3个神经元
# 用tf.Variable()标记参数可训练
# 使用seed使每次生成的随机数相同（方便教学，使大家结果都一致，在现实使用时不写seed）
w1 = tf.Variable(tf.random.truncated_normal([4, 3], stddev=0.1, seed=1))
b1 = tf.Variable(tf.random.truncated_normal([3], stddev=0.1, seed=1))

lr = 0.1  # 学习率为0.1
train_loss_results = []  # 将每轮的loss记录在此列表中，为后续画loss曲线提供数据
test_acc = []  # 将每轮的acc记录在此列表中，为后续画acc曲线提供数据
epoch = 500  # 循环500轮
loss_all = 0  # 每轮分4个step，loss_all记录四个step生成的4个loss的和

##########################################################################
#初始化参数
m_w, m_b = 0, 0
v_w, v_b = 0, 0
beta1, beta2 = 0.9, 0.999
delta_w, delta_b = 0, 0
global_step = 0
##########################################################################

# 训练部分
now_time = time.time()  ##2##
for epoch in range(epoch):  # 数据集级别的循环，每个epoch循环一次数据集
    for step, (x_train, y_train) in enumerate(train_db):  # batch级别的循环 ，每个step循环一个batch
 ##########################################################################       
        global_step += 1
 ##########################################################################       
        with tf.GradientTape() as tape:  # with结构记录梯度信息
            y = tf.matmul(x_train, w1) + b1  # 神经网络乘加运算
            y = tf.nn.softmax(y)  # 使输出y符合概率分布（此操作后与独热码同量级，可相减求loss）
            y_ = tf.one_hot(y_train, depth=3)  # 将标签值转换为独热码格式，方便计算loss和accuracy
            loss = tf.reduce_mean(tf.square(y_ - y))  # 采用均方误差损失函数mse = mean(sum(y-out)^2)
            loss_all += loss.numpy()  # 将每个step计算出的loss累加，为后续求loss平均值提供数据，这样计算的loss更准确
        # 计算loss对各个参数的梯度
        grads = tape.gradient(loss, [w1, b1])

##########################################################################
 # adam
        m_w = beta1 * m_w + (1 - beta1) * grads[0]
        m_b = beta1 * m_b + (1 - beta1) * grads[1]
        v_w = beta2 * v_w + (1 - beta2) * tf.square(grads[0])
        v_b = beta2 * v_b + (1 - beta2) * tf.square(grads[1])

        m_w_correction = m_w / (1 - tf.pow(beta1, int(global_step))) #修正后的一阶和二阶动量
        m_b_correction = m_b / (1 - tf.pow(beta1, int(global_step)))
        v_w_correction = v_w / (1 - tf.pow(beta2, int(global_step)))
        v_b_correction = v_b / (1 - tf.pow(beta2, int(global_step)))

        w1.assign_sub(lr * m_w_correction / tf.sqrt(v_w_correction))
        b1.assign_sub(lr * m_b_correction / tf.sqrt(v_b_correction))
##########################################################################

    # 每个epoch，打印loss信息
    print("Epoch {}, loss: {}".format(epoch, loss_all / 4))
    train_loss_results.append(loss_all / 4)  # 将4个step的loss求平均记录在此变量中
    loss_all = 0  # loss_all归零，为记录下一个epoch的loss做准备

    # 测试部分
    # total_correct为预测对的样本个数, total_number为测试的总样本数，将这两个变量都初始化为0
    total_correct, total_number = 0, 0
    for x_test, y_test in test_db:
        # 使用更新后的参数进行预测
        y = tf.matmul(x_test, w1) + b1
        y = tf.nn.softmax(y)
        pred = tf.argmax(y, axis=1)  # 返回y中最大值的索引，即预测的分类
        # 将pred转换为y_test的数据类型
        pred = tf.cast(pred, dtype=y_test.dtype)
        # 若分类正确，则correct=1，否则为0，将bool型的结果转换为int型
        correct = tf.cast(tf.equal(pred, y_test), dtype=tf.int32)
        # 将每个batch的correct数加起来
        correct = tf.reduce_sum(correct)
        # 将所有batch中的correct数加起来
        total_correct += int(correct)
        # total_number为测试的总样本数，也就是x_test的行数，shape[0]返回变量的行数
        total_number += x_test.shape[0]
    # 总的准确率等于total_correct/total_number
    acc = total_correct / total_number
    test_acc.append(acc)
    print("Test_acc:", acc)
    print("--------------------------")
total_time = time.time() - now_time  ##3##
print("total_time", total_time)  ##4##

# 绘制 loss 曲线
plt.title('Loss Function Curve')  # 图片标题
plt.xlabel('Epoch')  # x轴变量名称
plt.ylabel('Loss')  # y轴变量名称
plt.plot(train_loss_results, label="$Loss$")  # 逐点画出trian_loss_results值并连线，连线图标是Loss
plt.legend()  # 画出曲线图标
plt.show()  # 画出图像

# 绘制 Accuracy 曲线
plt.title('Acc Curve')  # 图片标题
plt.xlabel('Epoch')  # x轴变量名称
plt.ylabel('Acc')  # y轴变量名称
plt.plot(test_acc, label="$Accuracy$")  # 逐点画出test_acc值并连线，连线图标是Accuracy
plt.legend()
plt.show()

# 请将loss曲线、ACC曲线、total_time记录到 class2\优化器对比.docx  对比各优化器收敛情况

Epoch 0, loss: 0.219841156154871
Test_acc: 0.5333333333333333
--------------------------
Epoch 1, loss: 0.14480512216687202
Test_acc: 0.5333333333333333
--------------------------
Epoch 2, loss: 0.10274341143667698
Test_acc: 0.6666666666666666
--------------------------
Epoch 3, loss: 0.08922165259718895
Test_acc: 0.5333333333333333
--------------------------
Epoch 4, loss: 0.0860080998390913
Test_acc: 0.9
--------------------------
Epoch 5, loss: 0.06994969490915537
Test_acc: 0.8
--------------------------
Epoch 6, loss: 0.06724503170698881
Test_acc: 0.8
--------------------------
Epoch 7, loss: 0.061045361682772636
Test_acc: 1.0
--------------------------
Epoch 8, loss: 0.05573830287903547
Test_acc: 0.9333333333333333
--------------------------
Epoch 9, loss: 0.054052991792559624
Test_acc: 1.0
--------------------------
Epoch 10, loss: 0.0490921288728714
Test_acc: 1.0
--------------------------
Epoch 11, loss: 0.04825884848833084
Test_acc: 1.0
--------------------------
Epoch 12, loss: 0.04458624869585037
Test_acc: 1.0
--------------------------
Epoch 13, loss: 0.043710471130907536
Test_acc: 1.0
--------------------------
Epoch 14, loss: 0.04151816386729479
Test_acc: 1.0
--------------------------
Epoch 15, loss: 0.04042448848485947
Test_acc: 1.0
--------------------------
Epoch 16, loss: 0.03921938082203269
Test_acc: 1.0
--------------------------
Epoch 17, loss: 0.037702415604144335
Test_acc: 1.0
--------------------------
Epoch 18, loss: 0.03746768459677696
Test_acc: 1.0
--------------------------
Epoch 19, loss: 0.035456204786896706
Test_acc: 1.0
--------------------------
Epoch 20, loss: 0.03618265129625797
Test_acc: 1.0
--------------------------
Epoch 21, loss: 0.03353231865912676
Test_acc: 1.0
--------------------------
Epoch 22, loss: 0.03510467382147908
Test_acc: 1.0
--------------------------
Epoch 23, loss: 0.031799004413187504
Test_acc: 1.0
--------------------------
Epoch 24, loss: 0.034055131021887064
Test_acc: 1.0
--------------------------
Epoch 25, loss: 0.030265061650425196
Test_acc: 1.0
--------------------------
Epoch 26, loss: 0.032884009182453156
Test_acc: 1.0
--------------------------
Epoch 27, loss: 0.02898171078413725
Test_acc: 1.0
--------------------------
Epoch 28, loss: 0.03153973026201129
Test_acc: 1.0
--------------------------
Epoch 29, loss: 0.028038074262440205
Test_acc: 1.0
--------------------------
Epoch 30, loss: 0.03012115554884076
Test_acc: 1.0
--------------------------
Epoch 31, loss: 0.027460468467324972
Test_acc: 1.0
--------------------------
Epoch 32, loss: 0.028777056373655796
Test_acc: 1.0
--------------------------
Epoch 33, loss: 0.027132945135235786
Test_acc: 1.0
--------------------------
Epoch 34, loss: 0.02761527057737112
Test_acc: 1.0
--------------------------
Epoch 35, loss: 0.026862222235649824
Test_acc: 1.0
--------------------------
Epoch 36, loss: 0.02668716199696064
Test_acc: 1.0
--------------------------
Epoch 37, loss: 0.026503012515604496
Test_acc: 1.0
--------------------------
Epoch 38, loss: 0.025996222160756588
Test_acc: 1.0
--------------------------
Epoch 39, loss: 0.026023756247013807
Test_acc: 1.0
--------------------------
Epoch 40, loss: 0.02549735689535737
Test_acc: 1.0
--------------------------
Epoch 41, loss: 0.025481850374490023
Test_acc: 1.0
--------------------------
Epoch 42, loss: 0.025107113178819418
Test_acc: 1.0
--------------------------
Epoch 43, loss: 0.02495762286707759
Test_acc: 1.0
--------------------------
Epoch 44, loss: 0.02474382519721985
Test_acc: 1.0
--------------------------
Epoch 45, loss: 0.024502399377524853
Test_acc: 1.0
--------------------------
Epoch 46, loss: 0.024368189740926027
Test_acc: 1.0
--------------------------
Epoch 47, loss: 0.024121213238686323
Test_acc: 1.0
--------------------------
Epoch 48, loss: 0.023987011751160026
Test_acc: 1.0
--------------------------
Epoch 49, loss: 0.023787275422364473
Test_acc: 1.0
--------------------------
Epoch 50, loss: 0.023625574307516217
Test_acc: 1.0
--------------------------
Epoch 51, loss: 0.023471781983971596
Test_acc: 1.0
--------------------------
Epoch 52, loss: 0.023299274034798145
Test_acc: 1.0
--------------------------
Epoch 53, loss: 0.023163875099271536
Test_acc: 1.0
--------------------------
Epoch 54, loss: 0.023004963994026184
Test_acc: 1.0
--------------------------
Epoch 55, loss: 0.022868600441142917
Test_acc: 1.0
--------------------------
Epoch 56, loss: 0.022730717435479164
Test_acc: 1.0
--------------------------
Epoch 57, loss: 0.022593244444578886
Test_acc: 1.0
--------------------------
Epoch 58, loss: 0.02246865793131292
Test_acc: 1.0
--------------------------
Epoch 59, loss: 0.022338322130963206
Test_acc: 1.0
--------------------------
Epoch 60, loss: 0.022218456026166677
Test_acc: 1.0
--------------------------
Epoch 61, loss: 0.02209916291758418
Test_acc: 1.0
--------------------------
Epoch 62, loss: 0.02198244072496891
Test_acc: 1.0
--------------------------
Epoch 63, loss: 0.02187166642397642
Test_acc: 1.0
--------------------------
Epoch 64, loss: 0.021760689094662666
Test_acc: 1.0
--------------------------
Epoch 65, loss: 0.021654917625710368
Test_acc: 1.0
--------------------------
Epoch 66, loss: 0.021550980396568775
Test_acc: 1.0
--------------------------
Epoch 67, loss: 0.021449335850775242
Test_acc: 1.0
--------------------------
Epoch 68, loss: 0.021351256873458624
Test_acc: 1.0
--------------------------
Epoch 69, loss: 0.02125451061874628
Test_acc: 1.0
--------------------------
Epoch 70, loss: 0.02116080466657877
Test_acc: 1.0
--------------------------
Epoch 71, loss: 0.0210691608954221
Test_acc: 1.0
--------------------------
Epoch 72, loss: 0.020979427732527256
Test_acc: 1.0
--------------------------
Epoch 73, loss: 0.020892175612971187
Test_acc: 1.0
--------------------------
Epoch 74, loss: 0.020806559594348073
Test_acc: 1.0
--------------------------
Epoch 75, loss: 0.02072303812019527
Test_acc: 1.0
--------------------------
Epoch 76, loss: 0.020641361363232136
Test_acc: 1.0
--------------------------
Epoch 77, loss: 0.02056135074235499
Test_acc: 1.0
--------------------------
Epoch 78, loss: 0.020483192754909396
Test_acc: 1.0
--------------------------
Epoch 79, loss: 0.020406611962243915
Test_acc: 1.0
--------------------------
Epoch 80, loss: 0.020331630716100335
Test_acc: 1.0
--------------------------
Epoch 81, loss: 0.020258230855688453
Test_acc: 1.0
--------------------------
Epoch 82, loss: 0.020186285953968763
Test_acc: 1.0
--------------------------
Epoch 83, loss: 0.020115784369409084
Test_acc: 1.0
--------------------------
Epoch 84, loss: 0.02004669327288866
Test_acc: 1.0
--------------------------
Epoch 85, loss: 0.019978916039690375
Test_acc: 1.0
--------------------------
Epoch 86, loss: 0.019912483636289835
Test_acc: 1.0
--------------------------
Epoch 87, loss: 0.019847277086228132
Test_acc: 1.0
--------------------------
Epoch 88, loss: 0.01978331431746483
Test_acc: 1.0
--------------------------
Epoch 89, loss: 0.019720530603080988
Test_acc: 1.0
--------------------------
Epoch 90, loss: 0.019658871227875352
Test_acc: 1.0
--------------------------
Epoch 91, loss: 0.01959836296737194
Test_acc: 1.0
--------------------------
Epoch 92, loss: 0.019538898719474673
Test_acc: 1.0
--------------------------
Epoch 93, loss: 0.01948050269857049
Test_acc: 1.0
--------------------------
Epoch 94, loss: 0.019423116696998477
Test_acc: 1.0
--------------------------
Epoch 95, loss: 0.019366707652807236
Test_acc: 1.0
--------------------------
Epoch 96, loss: 0.019311273004859686
Test_acc: 1.0
--------------------------
Epoch 97, loss: 0.019256748724728823
Test_acc: 1.0
--------------------------
Epoch 98, loss: 0.019203145755454898
Test_acc: 1.0
--------------------------
Epoch 99, loss: 0.01915041427128017
Test_acc: 1.0
--------------------------
Epoch 100, loss: 0.019098545191809535
Test_acc: 1.0
--------------------------
Epoch 101, loss: 0.019047508016228676
Test_acc: 1.0
--------------------------
Epoch 102, loss: 0.018997267819941044
Test_acc: 1.0
--------------------------
Epoch 103, loss: 0.018947835080325603
Test_acc: 1.0
--------------------------
Epoch 104, loss: 0.018899151124060154
Test_acc: 1.0
--------------------------
Epoch 105, loss: 0.01885123853571713
Test_acc: 1.0
--------------------------
Epoch 106, loss: 0.018804020481184125
Test_acc: 1.0
--------------------------
Epoch 107, loss: 0.018757542595267296
Test_acc: 1.0
--------------------------
Epoch 108, loss: 0.01871174667030573
Test_acc: 1.0
--------------------------
Epoch 109, loss: 0.018666642485186458
Test_acc: 1.0
--------------------------
Epoch 110, loss: 0.018622159957885742
Test_acc: 1.0
--------------------------
Epoch 111, loss: 0.01857834868133068
Test_acc: 1.0
--------------------------
Epoch 112, loss: 0.01853515743277967
Test_acc: 1.0
--------------------------
Epoch 113, loss: 0.018492567585781217
Test_acc: 1.0
--------------------------
Epoch 114, loss: 0.01845061290077865
Test_acc: 1.0
--------------------------
Epoch 115, loss: 0.018409190233796835
Test_acc: 1.0
--------------------------
Epoch 116, loss: 0.018368377350270748
Test_acc: 1.0
--------------------------
Epoch 117, loss: 0.018328116508200765
Test_acc: 1.0
--------------------------
Epoch 118, loss: 0.018288369989022613
Test_acc: 1.0
--------------------------
Epoch 119, loss: 0.018249196698889136
Test_acc: 1.0
--------------------------
Epoch 120, loss: 0.018210522131994367
Test_acc: 1.0
--------------------------
Epoch 121, loss: 0.018172357231378555
Test_acc: 1.0
--------------------------
Epoch 122, loss: 0.018134709680452943
Test_acc: 1.0
--------------------------
Epoch 123, loss: 0.01809751708060503
Test_acc: 1.0
--------------------------
Epoch 124, loss: 0.018060836708173156
Test_acc: 1.0
--------------------------
Epoch 125, loss: 0.018024600110948086
Test_acc: 1.0
--------------------------
Epoch 126, loss: 0.01798883592709899
Test_acc: 1.0
--------------------------
Epoch 127, loss: 0.01795350224711001
Test_acc: 1.0
--------------------------
Epoch 128, loss: 0.0179186190944165
Test_acc: 1.0
--------------------------
Epoch 129, loss: 0.017884166911244392
Test_acc: 1.0
--------------------------
Epoch 130, loss: 0.01785012916661799
Test_acc: 1.0
--------------------------
Epoch 131, loss: 0.017816518899053335
Test_acc: 1.0
--------------------------
Epoch 132, loss: 0.01778329210355878
Test_acc: 1.0
--------------------------
Epoch 133, loss: 0.017750480910763144
Test_acc: 1.0
--------------------------
Epoch 134, loss: 0.01771805272437632
Test_acc: 1.0
--------------------------
Epoch 135, loss: 0.017685997067019343
Test_acc: 1.0
--------------------------
Epoch 136, loss: 0.017654340248554945
Test_acc: 1.0
--------------------------
Epoch 137, loss: 0.01762302708812058
Test_acc: 1.0
--------------------------
Epoch 138, loss: 0.017592082964256406
Test_acc: 1.0
--------------------------
Epoch 139, loss: 0.017561492044478655
Test_acc: 1.0
--------------------------
Epoch 140, loss: 0.01753125316463411
Test_acc: 1.0
--------------------------
Epoch 141, loss: 0.017501337686553597
Test_acc: 1.0
--------------------------
Epoch 142, loss: 0.01747178891673684
Test_acc: 1.0
--------------------------
Epoch 143, loss: 0.017442531185224652
Test_acc: 1.0
--------------------------
Epoch 144, loss: 0.0174136261921376
Test_acc: 1.0
--------------------------
Epoch 145, loss: 0.017385033192113042
Test_acc: 1.0
--------------------------
Epoch 146, loss: 0.017356734722852707
Test_acc: 1.0
--------------------------
Epoch 147, loss: 0.01732876803725958
Test_acc: 1.0
--------------------------
Epoch 148, loss: 0.017301088199019432
Test_acc: 1.0
--------------------------
Epoch 149, loss: 0.01727368892170489
Test_acc: 1.0
--------------------------
Epoch 150, loss: 0.01724661234766245
Test_acc: 1.0
--------------------------
Epoch 151, loss: 0.017219802364706993
Test_acc: 1.0
--------------------------
Epoch 152, loss: 0.017193270614370704
Test_acc: 1.0
--------------------------
Epoch 153, loss: 0.017167032696306705
Test_acc: 1.0
--------------------------
Epoch 154, loss: 0.017141057876870036
Test_acc: 1.0
--------------------------
Epoch 155, loss: 0.01711534452624619
Test_acc: 1.0
--------------------------
Epoch 156, loss: 0.017089904751628637
Test_acc: 1.0
--------------------------
Epoch 157, loss: 0.01706471759825945
Test_acc: 1.0
--------------------------
Epoch 158, loss: 0.017039785627275705
Test_acc: 1.0
--------------------------
Epoch 159, loss: 0.01701511791907251
Test_acc: 1.0
--------------------------
Epoch 160, loss: 0.01699067302979529
Test_acc: 1.0
--------------------------
Epoch 161, loss: 0.0169664837885648
Test_acc: 1.0
--------------------------
Epoch 162, loss: 0.016942543676123023
Test_acc: 1.0
--------------------------
Epoch 163, loss: 0.016918820096179843
Test_acc: 1.0
--------------------------
Epoch 164, loss: 0.016895336331799626
Test_acc: 1.0
--------------------------
Epoch 165, loss: 0.01687209028750658
Test_acc: 1.0
--------------------------
Epoch 166, loss: 0.0168490509968251
Test_acc: 1.0
--------------------------
Epoch 167, loss: 0.01682625967077911
Test_acc: 1.0
--------------------------
Epoch 168, loss: 0.016803660430014133
Test_acc: 1.0
--------------------------
Epoch 169, loss: 0.016781298676505685
Test_acc: 1.0
--------------------------
Epoch 170, loss: 0.016759122721850872
Test_acc: 1.0
--------------------------
Epoch 171, loss: 0.016737180994823575
Test_acc: 1.0
--------------------------
Epoch 172, loss: 0.01671542995609343
Test_acc: 1.0
--------------------------
Epoch 173, loss: 0.01669387868605554
Test_acc: 1.0
--------------------------
Epoch 174, loss: 0.016672548837959766
Test_acc: 1.0
--------------------------
Epoch 175, loss: 0.016651394311338663
Test_acc: 1.0
--------------------------
Epoch 176, loss: 0.016630453057587147
Test_acc: 1.0
--------------------------
Epoch 177, loss: 0.016609684331342578
Test_acc: 1.0
--------------------------
Epoch 178, loss: 0.016589115606620908
Test_acc: 1.0
--------------------------
Epoch 179, loss: 0.01656873431056738
Test_acc: 1.0
--------------------------
Epoch 180, loss: 0.016548529034480453
Test_acc: 1.0
--------------------------
Epoch 181, loss: 0.016528512351214886
Test_acc: 1.0
--------------------------
Epoch 182, loss: 0.01650867867283523
Test_acc: 1.0
--------------------------
Epoch 183, loss: 0.01648900588043034
Test_acc: 1.0
--------------------------
Epoch 184, loss: 0.016469520051032305
Test_acc: 1.0
--------------------------
Epoch 185, loss: 0.0164502018596977
Test_acc: 1.0
--------------------------
Epoch 186, loss: 0.016431050142273307
Test_acc: 1.0
--------------------------
Epoch 187, loss: 0.016412083758041263
Test_acc: 1.0
--------------------------
Epoch 188, loss: 0.01639324496500194
Test_acc: 1.0
--------------------------
Epoch 189, loss: 0.016374609898775816
Test_acc: 1.0
--------------------------
Epoch 190, loss: 0.01635610591620207
Test_acc: 1.0
--------------------------
Epoch 191, loss: 0.01633776957169175
Test_acc: 1.0
--------------------------
Epoch 192, loss: 0.016319590155035257
Test_acc: 1.0
--------------------------
Epoch 193, loss: 0.016301566967740655
Test_acc: 1.0
--------------------------
Epoch 194, loss: 0.01628370420075953
Test_acc: 1.0
--------------------------
Epoch 195, loss: 0.016265979502350092
Test_acc: 1.0
--------------------------
Epoch 196, loss: 0.016248409636318684
Test_acc: 1.0
--------------------------
Epoch 197, loss: 0.01623099227435887
Test_acc: 1.0
--------------------------
Epoch 198, loss: 0.016213689697906375
Test_acc: 1.0
--------------------------
Epoch 199, loss: 0.016196565004065633
Test_acc: 1.0
--------------------------
Epoch 200, loss: 0.016179573256522417
Test_acc: 1.0
--------------------------
Epoch 201, loss: 0.016162706771865487
Test_acc: 1.0
--------------------------
Epoch 202, loss: 0.01614598883315921
Test_acc: 1.0
--------------------------
Epoch 203, loss: 0.01612941548228264
Test_acc: 1.0
--------------------------
Epoch 204, loss: 0.016112952027469873
Test_acc: 1.0
--------------------------
Epoch 205, loss: 0.01609664922580123
Test_acc: 1.0
--------------------------
Epoch 206, loss: 0.016080467961728573
Test_acc: 1.0
--------------------------
Epoch 207, loss: 0.016064405906945467
Test_acc: 1.0
--------------------------
Epoch 208, loss: 0.016048494493588805
Test_acc: 1.0
--------------------------
Epoch 209, loss: 0.01603268599137664
Test_acc: 1.0
--------------------------
Epoch 210, loss: 0.016017016489058733
Test_acc: 1.0
--------------------------
Epoch 211, loss: 0.016001465497538447
Test_acc: 1.0
--------------------------
Epoch 212, loss: 0.01598604186438024
Test_acc: 1.0
--------------------------
Epoch 213, loss: 0.015970721375197172
Test_acc: 1.0
--------------------------
Epoch 214, loss: 0.015955543145537376
Test_acc: 1.0
--------------------------
Epoch 215, loss: 0.01594048086553812
Test_acc: 1.0
--------------------------
Epoch 216, loss: 0.015925514977425337
Test_acc: 1.0
--------------------------
Epoch 217, loss: 0.01591069041751325
Test_acc: 1.0
--------------------------
Epoch 218, loss: 0.015895966440439224
Test_acc: 1.0
--------------------------
Epoch 219, loss: 0.01588133885525167
Test_acc: 1.0
--------------------------
Epoch 220, loss: 0.015866854693740606
Test_acc: 1.0
--------------------------
Epoch 221, loss: 0.01585245318710804
Test_acc: 1.0
--------------------------
Epoch 222, loss: 0.0158381809014827
Test_acc: 1.0
--------------------------
Epoch 223, loss: 0.01582401292398572
Test_acc: 1.0
--------------------------
Epoch 224, loss: 0.015809949254617095
Test_acc: 1.0
--------------------------
Epoch 225, loss: 0.015795978251844645
Test_acc: 1.0
--------------------------
Epoch 226, loss: 0.01578212482854724
Test_acc: 1.0
--------------------------
Epoch 227, loss: 0.015768370823934674
Test_acc: 1.0
--------------------------
Epoch 228, loss: 0.015754723688587546
Test_acc: 1.0
--------------------------
Epoch 229, loss: 0.015741163166239858
Test_acc: 1.0
--------------------------
Epoch 230, loss: 0.0157277206890285
Test_acc: 1.0
--------------------------
Epoch 231, loss: 0.01571435621008277
Test_acc: 1.0
--------------------------
Epoch 232, loss: 0.015701111406087875
Test_acc: 1.0
--------------------------
Epoch 233, loss: 0.01568794925697148
Test_acc: 1.0
--------------------------
Epoch 234, loss: 0.015674879774451256
Test_acc: 1.0
--------------------------
Epoch 235, loss: 0.015661920653656125
Test_acc: 1.0
--------------------------
Epoch 236, loss: 0.015649036969989538
Test_acc: 1.0
--------------------------
Epoch 237, loss: 0.015636253403499722
Test_acc: 1.0
--------------------------
Epoch 238, loss: 0.01562356436625123
Test_acc: 1.0
--------------------------
Epoch 239, loss: 0.015610968694090843
Test_acc: 1.0
--------------------------
Epoch 240, loss: 0.015598442871123552
Test_acc: 1.0
--------------------------
Epoch 241, loss: 0.015586030436679721
Test_acc: 1.0
--------------------------
Epoch 242, loss: 0.015573688666336238
Test_acc: 1.0
--------------------------
Epoch 243, loss: 0.015561436070129275
Test_acc: 1.0
--------------------------
Epoch 244, loss: 0.015549276489764452
Test_acc: 1.0
--------------------------
Epoch 245, loss: 0.015537198167294264
Test_acc: 1.0
--------------------------
Epoch 246, loss: 0.015525204362347722
Test_acc: 1.0
--------------------------
Epoch 247, loss: 0.015513290418311954
Test_acc: 1.0
--------------------------
Epoch 248, loss: 0.015501468908041716
Test_acc: 1.0
--------------------------
Epoch 249, loss: 0.01548970746807754
Test_acc: 1.0
--------------------------
Epoch 250, loss: 0.015478059882298112
Test_acc: 1.0
--------------------------
Epoch 251, loss: 0.01546645408961922
Test_acc: 1.0
--------------------------
Epoch 252, loss: 0.015454950043931603
Test_acc: 1.0
--------------------------
Epoch 253, loss: 0.015443534590303898
Test_acc: 1.0
--------------------------
Epoch 254, loss: 0.01543217059224844
Test_acc: 1.0
--------------------------
Epoch 255, loss: 0.015420892275869846
Test_acc: 1.0
--------------------------
Epoch 256, loss: 0.015409694518893957
Test_acc: 1.0
--------------------------
Epoch 257, loss: 0.015398570918478072
Test_acc: 1.0
--------------------------
Epoch 258, loss: 0.015387511346489191
Test_acc: 1.0
--------------------------
Epoch 259, loss: 0.015376557945273817
Test_acc: 1.0
--------------------------
Epoch 260, loss: 0.01536564459092915
Test_acc: 1.0
--------------------------
Epoch 261, loss: 0.015354814822785556
Test_acc: 1.0
--------------------------
Epoch 262, loss: 0.015344054787419736
Test_acc: 1.0
--------------------------
Epoch 263, loss: 0.015333378803916276
Test_acc: 1.0
--------------------------
Epoch 264, loss: 0.015322742285206914
Test_acc: 1.0
--------------------------
Epoch 265, loss: 0.015312209143303335
Test_acc: 1.0
--------------------------
Epoch 266, loss: 0.015301719540730119
Test_acc: 1.0
--------------------------
Epoch 267, loss: 0.015291306306608021
Test_acc: 1.0
--------------------------
Epoch 268, loss: 0.015280970837920904
Test_acc: 1.0
--------------------------
Epoch 269, loss: 0.01527068973518908
Test_acc: 1.0
--------------------------
Epoch 270, loss: 0.015260480344295502
Test_acc: 1.0
--------------------------
Epoch 271, loss: 0.01525033253710717
Test_acc: 1.0
--------------------------
Epoch 272, loss: 0.015240265754982829
Test_acc: 1.0
--------------------------
Epoch 273, loss: 0.015230232733301818
Test_acc: 1.0
--------------------------
Epoch 274, loss: 0.015220295172184706
Test_acc: 1.0
--------------------------
Epoch 275, loss: 0.015210400801151991
Test_acc: 1.0
--------------------------
Epoch 276, loss: 0.015200570342130959
Test_acc: 1.0
--------------------------
Epoch 277, loss: 0.01519080065190792
Test_acc: 1.0
--------------------------
Epoch 278, loss: 0.015181119553744793
Test_acc: 1.0
--------------------------
Epoch 279, loss: 0.015171458013355732
Test_acc: 1.0
--------------------------
Epoch 280, loss: 0.015161880757659674
Test_acc: 1.0
--------------------------
Epoch 281, loss: 0.015152374980971217
Test_acc: 1.0
--------------------------
Epoch 282, loss: 0.015142898308113217
Test_acc: 1.0
--------------------------
Epoch 283, loss: 0.01513349951710552
Test_acc: 1.0
--------------------------
Epoch 284, loss: 0.015124151017516851
Test_acc: 1.0
--------------------------
Epoch 285, loss: 0.015114857698790729
Test_acc: 1.0
--------------------------
Epoch 286, loss: 0.015105637721717358
Test_acc: 1.0
--------------------------
Epoch 287, loss: 0.01509645814076066
Test_acc: 1.0
--------------------------
Epoch 288, loss: 0.015087342355400324
Test_acc: 1.0
--------------------------
Epoch 289, loss: 0.015078289317898452
Test_acc: 1.0
--------------------------
Epoch 290, loss: 0.015069274348206818
Test_acc: 1.0
--------------------------
Epoch 291, loss: 0.015060322941280901
Test_acc: 1.0
--------------------------
Epoch 292, loss: 0.015051410999149084
Test_acc: 1.0
--------------------------
Epoch 293, loss: 0.01504258718341589
Test_acc: 1.0
--------------------------
Epoch 294, loss: 0.015033786883577704
Test_acc: 1.0
--------------------------
Epoch 295, loss: 0.01502505224198103
Test_acc: 1.0
--------------------------
Epoch 296, loss: 0.015016362420283258
Test_acc: 1.0
--------------------------
Epoch 297, loss: 0.015007730107754469
Test_acc: 1.0
--------------------------
Epoch 298, loss: 0.014999139588326216
Test_acc: 1.0
--------------------------
Epoch 299, loss: 0.014990614261478186
Test_acc: 1.0
--------------------------
Epoch 300, loss: 0.014982132706791162
Test_acc: 1.0
--------------------------
Epoch 301, loss: 0.014973703306168318
Test_acc: 1.0
--------------------------
Epoch 302, loss: 0.014965309645049274
Test_acc: 1.0
--------------------------
Epoch 303, loss: 0.0149569904897362
Test_acc: 1.0
--------------------------
Epoch 304, loss: 0.01494870288297534
Test_acc: 1.0
--------------------------
Epoch 305, loss: 0.014940456254407763
Test_acc: 1.0
--------------------------
Epoch 306, loss: 0.014932268531993032
Test_acc: 1.0
--------------------------
Epoch 307, loss: 0.014924140414223075
Test_acc: 1.0
--------------------------
Epoch 308, loss: 0.014916043146513402
Test_acc: 1.0
--------------------------
Epoch 309, loss: 0.014907981734722853
Test_acc: 1.0
--------------------------
Epoch 310, loss: 0.01489999785553664
Test_acc: 1.0
--------------------------
Epoch 311, loss: 0.01489203330129385
Test_acc: 1.0
--------------------------
Epoch 312, loss: 0.014884120202623308
Test_acc: 1.0
--------------------------
Epoch 313, loss: 0.014876263216137886
Test_acc: 1.0
--------------------------
Epoch 314, loss: 0.014868438942357898
Test_acc: 1.0
--------------------------
Epoch 315, loss: 0.014860654715448618
Test_acc: 1.0
--------------------------
Epoch 316, loss: 0.014852941036224365
Test_acc: 1.0
--------------------------
Epoch 317, loss: 0.014845242723822594
Test_acc: 1.0
--------------------------
Epoch 318, loss: 0.014837592374533415
Test_acc: 1.0
--------------------------
Epoch 319, loss: 0.01482999639119953
Test_acc: 1.0
--------------------------
Epoch 320, loss: 0.014822438824921846
Test_acc: 1.0
--------------------------
Epoch 321, loss: 0.014814927708357573
Test_acc: 1.0
--------------------------
Epoch 322, loss: 0.014807443832978606
Test_acc: 1.0
--------------------------
Epoch 323, loss: 0.014800029923208058
Test_acc: 1.0
--------------------------
Epoch 324, loss: 0.014792609727010131
Test_acc: 1.0
--------------------------
Epoch 325, loss: 0.01478527463041246
Test_acc: 1.0
--------------------------
Epoch 326, loss: 0.014777959324419498
Test_acc: 1.0
--------------------------
Epoch 327, loss: 0.014770696870982647
Test_acc: 1.0
--------------------------
Epoch 328, loss: 0.014763465849682689
Test_acc: 1.0
--------------------------
Epoch 329, loss: 0.014756261254660785
Test_acc: 1.0
--------------------------
Epoch 330, loss: 0.0147491164971143
Test_acc: 1.0
--------------------------
Epoch 331, loss: 0.014742008643224835
Test_acc: 1.0
--------------------------
Epoch 332, loss: 0.014734935597516596
Test_acc: 1.0
--------------------------
Epoch 333, loss: 0.014727884205058217
Test_acc: 1.0
--------------------------
Epoch 334, loss: 0.014720908482559025
Test_acc: 1.0
--------------------------
Epoch 335, loss: 0.01471394021064043
Test_acc: 1.0
--------------------------
Epoch 336, loss: 0.014707002672366798
Test_acc: 1.0
--------------------------
Epoch 337, loss: 0.014700138824991882
Test_acc: 1.0
--------------------------
Epoch 338, loss: 0.014693273231387138
Test_acc: 1.0
--------------------------
Epoch 339, loss: 0.014686470851302147
Test_acc: 1.0
--------------------------
Epoch 340, loss: 0.014679691521450877
Test_acc: 1.0
--------------------------
Epoch 341, loss: 0.014672958292067051
Test_acc: 1.0
--------------------------
Epoch 342, loss: 0.014666252071037889
Test_acc: 1.0
--------------------------
Epoch 343, loss: 0.014659582288004458
Test_acc: 1.0
--------------------------
Epoch 344, loss: 0.014652953832410276
Test_acc: 1.0
--------------------------
Epoch 345, loss: 0.014646349591203034
Test_acc: 1.0
--------------------------
Epoch 346, loss: 0.014639788772910833
Test_acc: 1.0
--------------------------
Epoch 347, loss: 0.014633252052590251
Test_acc: 1.0
--------------------------
Epoch 348, loss: 0.014626753982156515
Test_acc: 1.0
--------------------------
Epoch 349, loss: 0.014620293397456408
Test_acc: 1.0
--------------------------
Epoch 350, loss: 0.014613865874707699
Test_acc: 1.0
--------------------------
Epoch 351, loss: 0.014607474207878113
Test_acc: 1.0
--------------------------
Epoch 352, loss: 0.014601112343370914
Test_acc: 1.0
--------------------------
Epoch 353, loss: 0.014594777021557093
Test_acc: 1.0
--------------------------
Epoch 354, loss: 0.014588481397368014
Test_acc: 1.0
--------------------------
Epoch 355, loss: 0.014582217670977116
Test_acc: 1.0
--------------------------
Epoch 356, loss: 0.014575970824807882
Test_acc: 1.0
--------------------------
Epoch 357, loss: 0.014569798018783331
Test_acc: 1.0
--------------------------
Epoch 358, loss: 0.014563607517629862
Test_acc: 1.0
--------------------------
Epoch 359, loss: 0.014557461603544652
Test_acc: 1.0
--------------------------
Epoch 360, loss: 0.014551374129951
Test_acc: 1.0
--------------------------
Epoch 361, loss: 0.014545290730893612
Test_acc: 1.0
--------------------------
Epoch 362, loss: 0.01453923317603767
Test_acc: 1.0
--------------------------
Epoch 363, loss: 0.01453323953319341
Test_acc: 1.0
--------------------------
Epoch 364, loss: 0.01452724460978061
Test_acc: 1.0
--------------------------
Epoch 365, loss: 0.014521293342113495
Test_acc: 1.0
--------------------------
Epoch 366, loss: 0.014515362679958344
Test_acc: 1.0
--------------------------
Epoch 367, loss: 0.014509474509395659
Test_acc: 1.0
--------------------------
Epoch 368, loss: 0.01450361032038927
Test_acc: 1.0
--------------------------
Epoch 369, loss: 0.014497770695015788
Test_acc: 1.0
--------------------------
Epoch 370, loss: 0.014491959009319544
Test_acc: 1.0
--------------------------
Epoch 371, loss: 0.014486184227280319
Test_acc: 1.0
--------------------------
Epoch 372, loss: 0.014480439480394125
Test_acc: 1.0
--------------------------
Epoch 373, loss: 0.01447470486164093
Test_acc: 1.0
--------------------------
Epoch 374, loss: 0.014469023328274488
Test_acc: 1.0
--------------------------
Epoch 375, loss: 0.014463347615674138
Test_acc: 1.0
--------------------------
Epoch 376, loss: 0.014457702403888106
Test_acc: 1.0
--------------------------
Epoch 377, loss: 0.014452096307650208
Test_acc: 1.0
--------------------------
Epoch 378, loss: 0.014446516055613756
Test_acc: 1.0
--------------------------
Epoch 379, loss: 0.014440959668718278
Test_acc: 1.0
--------------------------
Epoch 380, loss: 0.014435420278459787
Test_acc: 1.0
--------------------------
Epoch 381, loss: 0.014429924311116338
Test_acc: 1.0
--------------------------
Epoch 382, loss: 0.014424445922486484
Test_acc: 1.0
--------------------------
Epoch 383, loss: 0.014418993960134685
Test_acc: 1.0
--------------------------
Epoch 384, loss: 0.014413557713851333
Test_acc: 1.0
--------------------------
Epoch 385, loss: 0.014408168848603964
Test_acc: 1.0
--------------------------
Epoch 386, loss: 0.014402793603949249
Test_acc: 1.0
--------------------------
Epoch 387, loss: 0.014397447230294347
Test_acc: 1.0
--------------------------
Epoch 388, loss: 0.014392116339877248
Test_acc: 1.0
--------------------------
Epoch 389, loss: 0.014386835391633213
Test_acc: 1.0
--------------------------
Epoch 390, loss: 0.014381547924131155
Test_acc: 1.0
--------------------------
Epoch 391, loss: 0.014376316918060184
Test_acc: 1.0
--------------------------
Epoch 392, loss: 0.014371089520864189
Test_acc: 1.0
--------------------------
Epoch 393, loss: 0.014365885173901916
Test_acc: 1.0
--------------------------
Epoch 394, loss: 0.014360723085701466
Test_acc: 1.0
--------------------------
Epoch 395, loss: 0.014355575549416244
Test_acc: 1.0
--------------------------
Epoch 396, loss: 0.014350440818816423
Test_acc: 1.0
--------------------------
Epoch 397, loss: 0.014345347648486495
Test_acc: 1.0
--------------------------
Epoch 398, loss: 0.014340281370095909
Test_acc: 1.0
--------------------------
Epoch 399, loss: 0.014335206942632794
Test_acc: 1.0
--------------------------
Epoch 400, loss: 0.01433018024545163
Test_acc: 1.0
--------------------------
Epoch 401, loss: 0.01432518067304045
Test_acc: 1.0
--------------------------
Epoch 402, loss: 0.014320191810838878
Test_acc: 1.0
--------------------------
Epoch 403, loss: 0.014315234962850809
Test_acc: 1.0
--------------------------
Epoch 404, loss: 0.014310291735455394
Test_acc: 1.0
--------------------------
Epoch 405, loss: 0.014305362477898598
Test_acc: 1.0
--------------------------
Epoch 406, loss: 0.014300492941401899
Test_acc: 1.0
--------------------------
Epoch 407, loss: 0.014295608852989972
Test_acc: 1.0
--------------------------
Epoch 408, loss: 0.014290761551819742
Test_acc: 1.0
--------------------------
Epoch 409, loss: 0.01428593136370182
Test_acc: 1.0
--------------------------
Epoch 410, loss: 0.01428113307338208
Test_acc: 1.0
--------------------------
Epoch 411, loss: 0.014276345609687269
Test_acc: 1.0
--------------------------
Epoch 412, loss: 0.014271591557189822
Test_acc: 1.0
--------------------------
Epoch 413, loss: 0.014266852289438248
Test_acc: 1.0
--------------------------
Epoch 414, loss: 0.014262123266234994
Test_acc: 1.0
--------------------------
Epoch 415, loss: 0.014257429749704897
Test_acc: 1.0
--------------------------
Epoch 416, loss: 0.014252759981900454
Test_acc: 1.0
--------------------------
Epoch 417, loss: 0.014248088235035539
Test_acc: 1.0
--------------------------
Epoch 418, loss: 0.01424346084240824
Test_acc: 1.0
--------------------------
Epoch 419, loss: 0.014238850213587284
Test_acc: 1.0
--------------------------
Epoch 420, loss: 0.014234240516088903
Test_acc: 1.0
--------------------------
Epoch 421, loss: 0.014229685300961137
Test_acc: 1.0
--------------------------
Epoch 422, loss: 0.014225112274289131
Test_acc: 1.0
--------------------------
Epoch 423, loss: 0.014220598270185292
Test_acc: 1.0
--------------------------
Epoch 424, loss: 0.014216073090210557
Test_acc: 1.0
--------------------------
Epoch 425, loss: 0.014211573405191302
Test_acc: 1.0
--------------------------
Epoch 426, loss: 0.014207117143087089
Test_acc: 1.0
--------------------------
Epoch 427, loss: 0.014202660764567554
Test_acc: 1.0
--------------------------
Epoch 428, loss: 0.014198215096257627
Test_acc: 1.0
--------------------------
Epoch 429, loss: 0.014193806797266006
Test_acc: 1.0
--------------------------
Epoch 430, loss: 0.01418940513394773
Test_acc: 1.0
--------------------------
Epoch 431, loss: 0.014185025705955923
Test_acc: 1.0
--------------------------
Epoch 432, loss: 0.014180665253661573
Test_acc: 1.0
--------------------------
Epoch 433, loss: 0.014176331809721887
Test_acc: 1.0
--------------------------
Epoch 434, loss: 0.014172009890899062
Test_acc: 1.0
--------------------------
Epoch 435, loss: 0.014167708111926913
Test_acc: 1.0
--------------------------
Epoch 436, loss: 0.01416342135053128
Test_acc: 1.0
--------------------------
Epoch 437, loss: 0.014159158803522587
Test_acc: 1.0
--------------------------
Epoch 438, loss: 0.014154908247292042
Test_acc: 1.0
--------------------------
Epoch 439, loss: 0.01415068015921861
Test_acc: 1.0
--------------------------
Epoch 440, loss: 0.014146470814011991
Test_acc: 1.0
--------------------------
Epoch 441, loss: 0.014142273808829486
Test_acc: 1.0
--------------------------
Epoch 442, loss: 0.014138103928416967
Test_acc: 1.0
--------------------------
Epoch 443, loss: 0.014133934048004448
Test_acc: 1.0
--------------------------
Epoch 444, loss: 0.014129796763882041
Test_acc: 1.0
--------------------------
Epoch 445, loss: 0.014125673333182931
Test_acc: 1.0
--------------------------
Epoch 446, loss: 0.01412156573496759
Test_acc: 1.0
--------------------------
Epoch 447, loss: 0.01411747164092958
Test_acc: 1.0
--------------------------
Epoch 448, loss: 0.014113409677520394
Test_acc: 1.0
--------------------------
Epoch 449, loss: 0.01410935865715146
Test_acc: 1.0
--------------------------
Epoch 450, loss: 0.014105314621701837
Test_acc: 1.0
--------------------------
Epoch 451, loss: 0.01410129270516336
Test_acc: 1.0
--------------------------
Epoch 452, loss: 0.014097294886596501
Test_acc: 1.0
--------------------------
Epoch 453, loss: 0.01409330649767071
Test_acc: 1.0
--------------------------
Epoch 454, loss: 0.0140893270727247
Test_acc: 1.0
--------------------------
Epoch 455, loss: 0.014085383154451847
Test_acc: 1.0
--------------------------
Epoch 456, loss: 0.014081442495808005
Test_acc: 1.0
--------------------------
Epoch 457, loss: 0.014077521162107587
Test_acc: 1.0
--------------------------
Epoch 458, loss: 0.014073614031076431
Test_acc: 1.0
--------------------------
Epoch 459, loss: 0.014069720171391964
Test_acc: 1.0
--------------------------
Epoch 460, loss: 0.014065856579691172
Test_acc: 1.0
--------------------------
Epoch 461, loss: 0.014061981113627553
Test_acc: 1.0
--------------------------
Epoch 462, loss: 0.014058157452382147
Test_acc: 1.0
--------------------------
Epoch 463, loss: 0.014054326340556145
Test_acc: 1.0
--------------------------
Epoch 464, loss: 0.014050501631572843
Test_acc: 1.0
--------------------------
Epoch 465, loss: 0.01404671953059733
Test_acc: 1.0
--------------------------
Epoch 466, loss: 0.014042934169992805
Test_acc: 1.0
--------------------------
Epoch 467, loss: 0.014039159985259175
Test_acc: 1.0
--------------------------
Epoch 468, loss: 0.014035420725122094
Test_acc: 1.0
--------------------------
Epoch 469, loss: 0.014031685655936599
Test_acc: 1.0
--------------------------
Epoch 470, loss: 0.01402796688489616
Test_acc: 1.0
--------------------------
Epoch 471, loss: 0.014024262549355626
Test_acc: 1.0
--------------------------
Epoch 472, loss: 0.014020572765730321
Test_acc: 1.0
--------------------------
Epoch 473, loss: 0.014016890665516257
Test_acc: 1.0
--------------------------
Epoch 474, loss: 0.01401323382742703
Test_acc: 1.0
--------------------------
Epoch 475, loss: 0.014009584672749043
Test_acc: 1.0
--------------------------
Epoch 476, loss: 0.014005956589244306
Test_acc: 1.0
--------------------------
Epoch 477, loss: 0.014002344571053982
Test_acc: 1.0
--------------------------
Epoch 478, loss: 0.013998741749674082
Test_acc: 1.0
--------------------------
Epoch 479, loss: 0.013995146146044135
Test_acc: 1.0
--------------------------
Epoch 480, loss: 0.013991578598506749
Test_acc: 1.0
--------------------------
Epoch 481, loss: 0.013988004415296018
Test_acc: 1.0
--------------------------
Epoch 482, loss: 0.013984471908770502
Test_acc: 1.0
--------------------------
Epoch 483, loss: 0.013980937073938549
Test_acc: 1.0
--------------------------
Epoch 484, loss: 0.013977412017993629
Test_acc: 1.0
--------------------------
Epoch 485, loss: 0.013973913039080799
Test_acc: 1.0
--------------------------
Epoch 486, loss: 0.013970404514111578
Test_acc: 1.0
--------------------------
Epoch 487, loss: 0.013966937316581607
Test_acc: 1.0
--------------------------
Epoch 488, loss: 0.013963470933958888
Test_acc: 1.0
--------------------------
Epoch 489, loss: 0.013960018288344145
Test_acc: 1.0
--------------------------
Epoch 490, loss: 0.013956583105027676
Test_acc: 1.0
--------------------------
Epoch 491, loss: 0.01395316724665463
Test_acc: 1.0
--------------------------
Epoch 492, loss: 0.013949738815426826
Test_acc: 1.0
--------------------------
Epoch 493, loss: 0.013946352060884237
Test_acc: 1.0
--------------------------
Epoch 494, loss: 0.013942966354079545
Test_acc: 1.0
--------------------------
Epoch 495, loss: 0.013939589727669954
Test_acc: 1.0
--------------------------
Epoch 496, loss: 0.013936241273768246
Test_acc: 1.0
--------------------------
Epoch 497, loss: 0.01393288210965693
Test_acc: 1.0
--------------------------
Epoch 498, loss: 0.01392954622860998
Test_acc: 1.0
--------------------------
Epoch 499, loss: 0.01392624038271606
Test_acc: 1.0
--------------------------
total_time 60.744953870773315

你可能感兴趣的:(tensorflow,python,深度学习)

机器学习与深度学习间关系与区别 ℒℴѵℯ心·动ꦿ໊ོ꫞ 人工智能学习深度学习 python
一、机器学习概述定义机器学习（MachineLearning,ML）是一种通过数据驱动的方法，利用统计学和计算算法来训练模型，使计算机能够从数据中学习并自动进行预测或决策。机器学习通过分析大量数据样本，识别其中的模式和规律，从而对新的数据进行判断。其核心在于通过训练过程，让模型不断优化和提升其预测准确性。主要类型1.监督学习（SupervisedLearning）监督学习是指在训练数据集中包含输入
理解Gunicorn：Python WSGI服务器的基石范范0825 ipython linux 运维
理解Gunicorn：PythonWSGI服务器的基石介绍Gunicorn，全称GreenUnicorn，是一个为PythonWSGI（WebServerGatewayInterface）应用设计的高效、轻量级HTTP服务器。作为PythonWeb应用部署的常用工具，Gunicorn以其高性能和易用性著称。本文将介绍Gunicorn的基本概念、安装和配置，帮助初学者快速上手。1.什么是Gunico
Python数据分析与可视化实战指南 William数据分析 python python 数据
在数据驱动的时代，Python因其简洁的语法、强大的库生态系统以及活跃的社区，成为了数据分析与可视化的首选语言。本文将通过一个详细的案例，带领大家学习如何使用Python进行数据分析，并通过可视化来直观呈现分析结果。一、环境准备1.1安装必要库在开始数据分析和可视化之前，我们需要安装一些常用的库。主要包括pandas、numpy、matplotlib和seaborn等。这些库分别用于数据处理、数学
python os.environ 江湖偌大 python 深度学习
os.environ['TF_CPP_MIN_LOG_LEVEL']='0'#默认值，输出所有信息os.environ['TF_CPP_MIN_LOG_LEVEL']='1'#屏蔽通知信息（INFO）os.environ['TF_CPP_MIN_LOG_LEVEL']='2'#屏蔽通知信息和警告信息（INFO\WARNING）os.environ['TF_CPP_MIN_LOG_LEVEL']='
Python中os.environ基本介绍及使用方法鹤冲天Pro #Python python 服务器开发语言
文章目录python中os.environos.environ简介os.environ进行环境变量的增删改查python中os.environ的使用详解1.简介2.key字段详解2.1常见key字段3.os.environ.get()用法4.环境变量的增删改查和判断是否存在4.1新增环境变量4.2更新环境变量4.3获取环境变量4.4删除环境变量4.5判断环境变量是否存在python中os.envi
Pyecharts数据可视化大屏：打造沉浸式数据分析体验我的运维人生信息可视化数据分析数据挖掘运维开发技术共享
Pyecharts数据可视化大屏：打造沉浸式数据分析体验在当今这个数据驱动的时代，如何将海量数据以直观、生动的方式展现出来，成为了数据分析师和企业决策者关注的焦点。Pyecharts，作为一款基于Python的开源数据可视化库，凭借其丰富的图表类型、灵活的配置选项以及高度的定制化能力，成为了构建数据可视化大屏的理想选择。本文将深入探讨如何利用Pyecharts打造数据可视化大屏，并通过实际代码案例
Python教程：一文了解使用Python处理XPath 旦莫 Python进阶 python 开发语言
目录1.环境准备1.1安装lxml1.2验证安装2.XPath基础2.1什么是XPath？2.2XPath语法2.3示例XML文档3.使用lxml解析XML3.1解析XML文档3.2查看解析结果4.XPath查询4.1基本路径查询4.2使用属性查询4.3查询多个节点5.XPath的高级用法5.1使用逻辑运算符5.2使用函数6.实战案例6.1从网页抓取数据6.1.1安装Requests库6.1.2代
python os.environ_python os.environ 读取和设置环境变量 weixin_39605414 python os.environ
>>>importos>>>os.environ.keys()['LC_NUMERIC','GOPATH','GOROOT','GOBIN','LESSOPEN','SSH_CLIENT','LOGNAME','USER','HOME','LC_PAPER','PATH','DISPLAY','LANG','TERM','SHELL','J2REDIR','LC_MONETARY','QT_QPA
将cmd中命令输出保存为txt文本文件落难Coder Windows cmd window
最近深度学习本地的训练中我们常常要在命令行中运行自己的代码，无可厚非，我们有必要保存我们的炼丹结果，但是复制命令行输出到txt是非常麻烦的，其实Windows下的命令行为我们提供了相应的操作。其基本的调用格式就是：运行指令>输出到的文件名称或者具体保存路径测试下，我打开cmd并且ping一下百度：pingwww.baidu.com>./data.txt看下相同目录下data.txt的输出：如果你再
使用Faiss进行高效相似度搜索 llzwxh888 faiss python
在现代AI应用中，快速和高效的相似度搜索是至关重要的。Faiss（FacebookAISimilaritySearch）是一个专门用于快速相似度搜索和聚类的库，特别适用于高维向量。本文将介绍如何使用Faiss来进行相似度搜索，并结合Python代码演示其基本用法。什么是Faiss？Faiss是一个由FacebookAIResearch团队开发的开源库，主要用于高维向量的相似性搜索和聚类。Faiss
python是什么意思中文-在python中%是什么意思编程大乐趣
Python中%有两种：1、数值运算：%代表取模，返回除法的余数。如：>>>7%212、%操作符（字符串格式化，stringformatting），说明如下：%[(name)][flags][width].[precision]typecode(name)为命名flags可以有+，-，''或0。+表示右对齐。-表示左对齐。''为一个空格，表示在正数的左侧填充一个空格，从而与负数对齐。0表示使用0填
Day1笔记-Python简介&标识符和关键字&输入输出 ~在杰难逃~ Python python 开发语言大数据数据分析数据挖掘
大家好，从今天开始呢，杰哥开展一个新的专栏，当然，数据分析部分也会不定时更新的，这个新的专栏主要是讲解一些Python的基础语法和知识，帮助0基础的小伙伴入门和学习Python，感兴趣的小伙伴可以开始认真学习啦！一、Python简介【了解】1.计算机工作原理编程语言就是用来定义计算机程序的形式语言。我们通过编程语言来编写程序代码，再通过语言处理程序执行向计算机发送指令，让计算机完成对应的工作，编程
python八股文面试题分享及解析(1) Shawn________ python
#1.'''a=1b=2不用中间变量交换a和b'''#1.a=1b=2a,b=b,aprint(a)print(b)结果：21#2.ll=[]foriinrange(3):ll.append({'num':i})print(11)结果:#[{'num':0},{'num':1},{'num':2}]#3.kk=[]a={'num':0}foriinrange(3):#0,12#可变类型，不仅仅改变
每日算法&面试题，大厂特训二十八天——第二十天（树）肥学 ⚡算法题⚡面试题每日精进 java 算法数据结构
目录标题导读算法特训二十八天面试题点击直接资料领取导读肥友们为了更好的去帮助新同学适应算法和面试题，最近我们开始进行专项突击一步一步来。上一期我们完成了动态规划二十一天现在我们进行下一项对各类算法进行二十八天的一个小总结。还在等什么快来一起肥学进行二十八天挑战吧！！特别介绍小白练手专栏，适合刚入手的新人欢迎订阅编程小白进阶python有趣练手项目里面包括了像《机器人尬聊》《恶搞程序》这样的有趣文章
Python快速入门 —— 第三节：类与对象孤华暗香 Python快速入门 python 开发语言
第三节：类与对象目标：了解面向对象编程的基础概念，并学会如何定义类和创建对象。内容：类与对象：定义类：class关键字。类的构造函数：__init__()。类的属性和方法。对象的创建与使用。示例：classStudent:def__init__(self,name,age,major):self.name&#
pyecharts——绘制柱形图折线图 2224070247 信息可视化 python java 数据可视化
一、pyecharts概述自2013年6月百度EFE(ExcellentFrontEnd）数据可视化团队研发的ECharts1.0发布到GitHub网站以来，ECharts一直备受业界权威的关注并获得广泛好评，成为目前成熟且流行的数据可视化图表工具，被应用到诸多数据可视化的开发领域。Python作为数据分析领域最受欢迎的语言，也加入ECharts的使用行列，并研发出方便Python开发者使用的数据
Python 实现图片裁剪（附代码） | Python工具剑客阿良_ALiang
前言本文提供将图片按照自定义尺寸进行裁剪的工具方法，一如既往的实用主义。环境依赖ffmpeg环境安装，可以参考我的另一篇文章：windowsffmpeg安装部署_阿良的博客-CSDN博客本文主要使用到的不是ffmpeg，而是ffprobe也在上面这篇文章中的zip包中。ffmpy安装：pipinstallffmpy-ihttps://pypi.douban.com/simple代码不废话了，上代码
【华为OD技术面试真题 - 技术面】- python八股文真题题库（4) 算法大师华为od 面试 python
华为OD面试真题精选专栏：华为OD面试真题精选目录:2024华为OD面试手撕代码真题目录以及八股文真题目录文章目录华为OD面试真题精选**1.Python中的`with`**用途和功能自动资源管理示例：文件操作上下文管理协议示例代码工作流程解析优点2.\_\_new\_\_和**\_\_init\_\_**区别__new____init__区别总结3.**切片（Slicing）操作**基本切片语法
python os 环境变量 CV矿工 python 开发语言 numpy
环境变量：环境变量是程序和操作系统之间的通信方式。有些字符不宜明文写进代码里，比如数据库密码，个人账户密码，如果写进自己本机的环境变量里，程序用的时候通过os.environ.get（）取出来就行了。os.environ是一个环境变量的字典。环境变量的相关操作importos"""设置/修改环境变量：os.environ[‘环境变量名称’]=‘环境变量值’#其中key和value均为string类
Python爬虫解析工具之xpath使用详解 eqa11 python 爬虫开发语言
文章目录Python爬虫解析工具之xpath使用详解一、引言二、环境准备1、插件安装2、依赖库安装三、xpath语法详解1、路径表达式2、通配符3、谓语4、常用函数四、xpath在Python代码中的使用1、文档树的创建2、使用xpath表达式3、获取元素内容和属性五、总结Python爬虫解析工具之xpath使用详解一、引言在Python爬虫开发中，数据提取是一个至关重要的环节。xpath作为一门
【华为OD技术面试真题 - 技术面】- python八股文真题题库（1）算法大师华为od 面试 python
华为OD面试真题精选专栏：华为OD面试真题精选目录:2024华为OD面试手撕代码真题目录以及八股文真题目录文章目录华为OD面试真题精选1.数据预处理流程数据预处理的主要步骤工具和库2.介绍线性回归、逻辑回归模型线性回归（LinearRegression）模型形式：关键点：逻辑回归（LogisticRegression）模型形式：关键点：参数估计与评估：3.python浅拷贝及深拷贝浅拷贝（Shal
nosql数据库技术与应用知识点皆过客，揽星河 NoSQL nosql 数据库大数据数据分析数据结构非关系型数据库
Nosql知识回顾大数据处理流程数据采集(flume、爬虫、传感器)数据存储(本门课程NoSQL所处的阶段)Hdfs、MongoDB、HBase等数据清洗(入仓)Hive等数据处理、分析(Spark、Flink等)数据可视化数据挖掘、机器学习应用(Python、SparkMLlib等)大数据时代存储的挑战(三高)高并发(同一时间很多人访问)高扩展(要求随时根据需求扩展存储)高效率(要求读写速度快)
《Python数据分析实战终极指南》 xjt921122 python 数据分析开发语言
对于分析师来说，大家在学习Python数据分析的路上，多多少少都遇到过很多大坑**，有关于技能和思维的**：Excel已经没办法处理现有的数据量了，应该学Python吗？找了一大堆Python和Pandas的资料来学习，为什么自己动手就懵了？跟着比赛类公开数据分析案例练了很久，为什么当自己面对数据需求还是只会数据处理而没有分析思路？学了对比、细分、聚类分析，也会用PEST、波特五力这类分析法，为啥
Python中深拷贝与浅拷贝的区别 yuxiaoyu.
转自：http://blog.csdn.net/u014745194/article/details/70271868定义：在Python中对象的赋值其实就是对象的引用。当创建一个对象，把它赋值给另一个变量的时候，python并没有拷贝这个对象，只是拷贝了这个对象的引用而已。浅拷贝：拷贝了最外围的对象本身，内部的元素都只是拷贝了一个引用而已。也就是，把对象复制一遍，但是该对象中引用的其他对象我不复
Python开发常用的三方模块如下：换个网名有点难 python 开发语言
Python是一门功能强大的编程语言，拥有丰富的第三方库，这些库为开发者提供了极大的便利。以下是100个常用的Python库，涵盖了多个领域：1、NumPy，用于科学计算的基础库。2、Pandas，提供数据结构和数据分析工具。3、Matplotlib，一个绘图库。4、Scikit-learn，机器学习库。5、SciPy，用于数学、科学和工程的库。6、TensorFlow，由Google开发的开源机
Python编译器鹿鹿~ Python编译器 Python python 开发语言后端
嘿嘿嘿我又来了啊有些小盆友可能不知道Python其实是有编译器的，也就是PyCharm。你们可能会问到这个是干嘛的又不可以吃也不可以穿好像没有什么用，其实你还说对了这个还真的不可以吃也不可以穿，但是它用来干嘛的呢。用来编译你所打出的代码进行运行（可能这里说的有点不对但是只是个人认为）现在我们来说说PyCharm是用来干嘛的。PyCharm是一种PythonIDE，带有一整套可以帮助用户在使用Pyt
一文掌握python面向对象魔术方法（二）程序员neil python python 开发语言
接上篇：一文掌握python面向对象魔术方法（一）-CSDN博客目录六、迭代和序列化：1、__iter__(self):定义迭代器，使得类可以被for循环迭代。2、__getitem__(self,key):定义索引操作，如obj[key]。3、__setitem__(self,key,value):定义赋值操作，如obj[key]=value。4、__delitem__(self,key):定义
一文掌握python常用的list（列表）操作程序员neil python python 开发语言
目录一、创建列表1.直接创建列表：2.使用list()构造器3.使用列表推导式4.创建空列表二、访问列表元素1.列表支持通过索引访问元素，索引从0开始：2.还可以使用切片操作访问列表的一部分：三、修改列表元素四、添加元素1.append()：在末尾添加元素2.insert()：在指定位置插入元素五、删除元素1.del：删除指定位置的元素2.remove()：删除指定值的第一个匹配项3.pop()：
Python实现简单的机器学习算法 master_chenchengg python python 办公效率 python开发 IT
Python实现简单的机器学习算法开篇：初探机器学习的奇妙之旅搭建环境：一切从安装开始必备工具箱第一步：安装Anaconda和JupyterNotebook小贴士：如何配置Python环境变量算法初体验：从零开始的Python机器学习线性回归：让数据说话数据准备：从哪里找数据编码实战：Python实现线性回归模型评估：如何判断模型好坏逻辑回归：从分类开始理论入门：什么是逻辑回归代码实现：使用skl
python中的深拷贝与浅拷贝 anshejd70787 python
深拷贝和浅拷贝浅拷贝的时候，修改原来的对象，浅拷贝的对象不会发生改变。1、对象的赋值对象的赋值实际上是对象之间的引用：当创建一个对象，然后将这个对象赋值给另外一个变量的时候，python并没有拷贝这个对象，而只是拷贝了这个对象的引用。当对对象做赋值或者是参数传递或者作为返回值的时候，总是传递原始对象的引用，而不是一个副本。如下所示：>>>aList=["kel","abc",123]>>>bLis
Spring4.1新特性——Spring MVC增强 jinnianshilongnian spring 4.1
目录 Spring4.1新特性——综述 Spring4.1新特性——Spring核心部分及其他 Spring4.1新特性——Spring缓存框架增强 Spring4.1新特性——异步调用和事件机制的异常处理 Spring4.1新特性——数据库集成测试脚本初始化 Spring4.1新特性——Spring MVC增强 Spring4.1新特性——页面自动化测试框架Spring MVC T
mysql 性能查询优化 annan211 java sql 优化 mysql 应用服务器
1 时间到底花在哪了？ mysql在执行查询的时候需要执行一系列的子任务，这些子任务包含了整个查询周期最重要的阶段，这其中包含了大量为了检索数据列到存储引擎的调用以及调用后的数据处理，包括排序、分组等。在完成这些任务的时候，查询需要在不同的地方花费时间，包括网络、cpu计算、生成统计信息和执行计划、锁等待等。尤其是向底层存储引擎检索数据的调用操作。这些调用需要在内存操
windows系统配置 cherishLC windows
删除Hiberfil.sys ：使用命令powercfg -h off 关闭休眠功能即可： http://jingyan.baidu.com/article/f3ad7d0fc0992e09c2345b51.html 类似的还有pagefile.sys msconfig 配置启动项 shutdown 定时关机 ipconfig 查看网络配置 ipconfig /flushdns
人体的排毒时间 Array_06 工作
======================== || 人体的排毒时间是什么时候？|| ======================== 转载于： http://zhidao.baidu.com/link?url=ibaGlicVslAQhVdWWVevU4TMjhiKaNBWCpZ1NS6igCQ78EkNJZFsEjCjl3T5EdXU9SaPg04bh8MbY1bR
ZooKeeper cugfy zookeeper
Zookeeper是一个高性能，分布式的，开源分布式应用协调服务。它提供了简单原始的功能，分布式应用可以基于它实现更高级的服务，比如同步，配置管理，集群管理，名空间。它被设计为易于编程，使用文件系统目录树作为数据模型。服务端跑在java上，提供java和C的客户端API。 Zookeeper是Google的Chubby一个开源的实现，是高有效和可靠的协同工作系统，Zookeeper能够用来lea
网络爬虫的乱码处理随意而生爬虫网络
下边简单总结下关于网络爬虫的乱码处理。注意，这里不仅是中文乱码，还包括一些如日文、韩文、俄文、藏文之类的乱码处理，因为他们的解决方式是一致的，故在此统一说明。网络爬虫，有两种选择，一是选择nutch、hetriex，二是自写爬虫，两者在处理乱码时，原理是一致的，但前者处理乱码时，要看懂源码后进行修改才可以，所以要废劲一些；而后者更自由方便，可以在编码处理
Xcode常用快捷键张亚雄 xcode
一、总结的常用命令：隐藏xcode command+h 退出xcode command+q 关闭窗口 command+w 关闭所有窗口 command+option+w 关闭当前
mongoDB索引操作 adminjun mongodb 索引
一、索引基础： MongoDB的索引几乎与传统的关系型数据库一模一样，这其中也包括一些基本的优化技巧。下面是创建索引的命令： > db.test.ensureIndex({"username":1}) 可以通过下面的名称查看索引是否已经成功建立： &nbs
成都软件园实习那些话 aijuans 成都软件园实习
无聊之中，翻了一下日志，发现上一篇经历是很久以前的事了，悔过~~ 　　断断续续离开了学校快一年了，习惯了那里一天天的幼稚、成长的环境，到这里有点与世隔绝的感觉。不过还好，那是刚到这里时的想法，现在感觉在这挺好，不管怎么样，最要感谢的还是老师能给这么好的一次催化成长的机会，在这里确实看到了好多好多能想到或想不到的东西。　　都说在外面和学校相比最明显的差距就是与人相处比较困难，因为在外面每个人都
Linux下FTP服务器安装及配置 ayaoxinchao linux FTP服务器 vsftp
检测是否安装了FTP [root@localhost ~]# rpm -q vsftpd 如果未安装：package vsftpd is not installed 安装了则显示：vsftpd-2.0.5-28.el5累死的版本信息安装FTP 运行yum install vsftpd命令，如[root@localhost ~]# yum install vsf
使用mongo-java-driver获取文档id和查找文档 BigBird2012 driver
注：本文所有代码都使用的mongo-java-driver实现。在MongoDB中，一个集合（collection）在概念上就类似我们SQL数据库中的表（Table），这个集合包含了一系列文档（document）。一个DBObject对象表示我们想添加到集合（collection）中的一个文档（document），MongoDB会自动为我们创建的每个文档添加一个id，这个id在
JSONObject以及json串 bijian1013 json JSONObject
一.JAR包简介要使程序可以运行必须引入JSON-lib包，JSON-lib包同时依赖于以下的JAR包： 1.commons-lang-2.0.jar 2.commons-beanutils-1.7.0.jar 3.commons-collections-3.1.jar &n
[Zookeeper学习笔记之三]Zookeeper实例创建和会话建立的异步特性 bit1129 zookeeper
为了说明问题，看个简单的代码， import org.apache.zookeeper.*; import java.io.IOException; import java.util.concurrent.CountDownLatch; import java.util.concurrent.ThreadLocal
【Scala十二】Scala核心六：Trait bit1129 scala
Traits are a fundamental unit of code reuse in Scala. A trait encapsulates method and field definitions, which can then be reused by mixing them into classes. Unlike class inheritance, in which each c
weblogic version 10.3破解 ronin47 weblogic
版本：WebLogic Server 10.3 说明：%DOMAIN_HOME%：指WebLogic Server 域(Domain）目录例如我的做测试的域的根目录 DOMAIN_HOME=D:/Weblogic/Middleware/user_projects/domains/base_domain 1.为了保证操作安全，备份%DOMAIN_HOME%/security/Defa
求第n个斐波那契数 BrokenDreams
今天看到群友发的一个问题：写一个小程序打印第n个斐波那契数。自己试了下，搞了好久。。。基础要加强了。 &nbs
读《研磨设计模式》-代码笔记-访问者模式-Visitor bylijinnan java 设计模式
声明：本文只为方便我个人查阅和理解，详细的分析以及源代码请移步原作者的博客http://chjavach.iteye.com/ import java.util.ArrayList; import java.util.List; interface IVisitor { //第二次分派，Visitor调用Element void visitConcret
MatConvNet的excise 3改为网络配置文件形式 cherishLC matlab
MatConvNet为vlFeat作者写的matlab下的卷积神经网络工具包，可以使用GPU。主页： http://www.vlfeat.org/matconvnet/ 教程： http://www.robots.ox.ac.uk/~vgg/practicals/cnn/index.html 注意：需要下载新版的MatConvNet替换掉教程中工具包中的matconvnet： http
ZK Timeout再讨论 chenchao051 zookeeper timeout hbase
http://crazyjvm.iteye.com/blog/1693757 文中提到相关超时问题，但是又出现了一个问题，我把min和max都设置成了180000，但是仍然出现了以下的异常信息： Client session timed out, have not heard from server in 154339ms for sessionid 0x13a3f7732340003
CASE WHEN 用法介绍 daizj sql group by case when
CASE WHEN 用法介绍 1. CASE WHEN 表达式有两种形式 --简单Case函数 CASE sex WHEN '1' THEN '男' WHEN '2' THEN '女' ELSE '其他' END --Case搜索函数 CASE WHEN sex = '1' THEN
PHP技巧汇总:提高PHP性能的53个技巧 dcj3sjt126com PHP
PHP技巧汇总:提高PHP性能的53个技巧　　用单引号代替双引号来包含字符串，这样做会更快一些。因为PHP会在双引号包围的字符串中搜寻变量，　　单引号则不会，注意：只有echo能这么做，它是一种可以把多个字符串当作参数的函数译注：　　PHP手册中说echo是语言结构，不是真正的函数，故把函数加上了双引号)。　　1、如果能将类的方法定义成static，就尽量定义成static，它的速度会提升将近4倍
Yii框架中CGridView的使用方法以及详细示例 dcj3sjt126com yii
CGridView显示一个数据项的列表中的一个表。表中的每一行代表一个数据项的数据,和一个列通常代表一个属性的物品(一些列可能对应于复杂的表达式的属性或静态文本)。　　CGridView既支持排序和分页的数据项。排序和分页可以在AJAX模式或正常的页面请求。使用CGridView的一个好处是,当用户浏览器禁用JavaScript,排序和分页自动退化普通页面请求和仍然正常运行。实例代码如下：
Maven项目打包成可执行Jar文件 dyy_gusi assembly
Maven项目打包成可执行Jar文件在使用Maven完成项目以后，如果是需要打包成可执行的Jar文件，我们通过eclipse的导出很麻烦，还得指定入口文件的位置，还得说明依赖的jar包，既然都使用Maven了，很重要的一个目的就是让这些繁琐的操作简单。我们可以通过插件完成这项工作，使用assembly插件。具体使用方式如下： 1、在项目中加入插件的依赖： <plugin>
php常见错误 geeksun PHP
1. kevent() reported that connect() failed (61: Connection refused) while connecting to upstream, client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", upstream: "fastc
修改linux的用户名 hongtoushizi linux change password
Change Linux Username 更改Linux用户名，需要修改4个系统的文件： /etc/passwd /etc/shadow /etc/group /etc/gshadow 古老/传统的方法是使用vi去直接修改，但是这有安全隐患（具体可自己搜一下），所以后来改成使用这些命令去代替： vipw vipw -s vigr vigr -s 具体的操作顺
第五章常用Lua开发库1-redis、mysql、http客户端 jinnianshilongnian nginx lua
对于开发来说需要有好的生态开发库来辅助我们快速开发，而Lua中也有大多数我们需要的第三方开发库如Redis、Memcached、Mysql、Http客户端、JSON、模板引擎等。一些常见的Lua库可以在github上搜索，https://github.com/search?utf8=%E2%9C%93&q=lua+resty。 Redis客户端 lua-resty-r
zkClient 监控机制实现 liyonghui160com zkClient 监控机制实现
直接使用zk的api实现业务功能比较繁琐。因为要处理session loss，session expire等异常，在发生这些异常后进行重连。又因为ZK的watcher是一次性的，如果要基于wather实现发布/订阅模式，还要自己包装一下，将一次性订阅包装成持久订阅。另外如果要使用抽象级别更高的功能，比如分布式锁，leader选举
在Mysql 众多表中查找一个表名或者字段名的 SQL 语句 pda158 mysql
在Mysql 众多表中查找一个表名或者字段名的 SQL 语句：　　方法一：SELECT table_name, column_name from information_schema.columns WHERE column_name LIKE 'Name'; 　　方法二：SELECT column_name from information_schema.colum
程序员对英语的依赖 Smile.zeng 英语程序猿
1、程序员最基本的技能，至少要能写得出代码，当我们还在为建立类的时候思考用什么单词发牢骚的时候，英语与别人的差距就直接表现出来咯。 2、程序员最起码能认识开发工具里的英语单词，不然怎么知道使用这些开发工具。 3、进阶一点，就是能读懂别人的代码，有利于我们学习人家的思路和技术。 4、写的程序至少能有一定的可读性，至少要人别人能懂吧... 以上一些问题，充分说明了英语对程序猿的重要性。骚年
Oracle学习笔记(8) 使用PLSQL编写触发器 vipbooks oracle sql 编程活动 Access
时间过得真快啊，转眼就到了Oracle学习笔记的最后个章节了，通过前面七章的学习大家应该对Oracle编程有了一定了了解了吧，这东东如果一段时间不用很快就会忘记了，所以我会把自己学习过的东西做好详细的笔记，用到的时候可以随时查找，马上上手！希望这些笔记能对大家有些帮助！这是第八章的学习笔记，学习完第七章的子程序和包之后