深度学习的数据预处理(Data Preprocessing),权值初始化(weight initialization),损失函数(loss functions)

数据预处理(data preprocessing)

注意: 只计算training data的均值; 训练的时候减去这个均值做训练; Val/Test阶段也是减去这个均值;

Mean subtraction

X: NxD (N is number of data; D is number of dimenssionality)

X -=np.mean(X,axis=0)

对于图片:

# Preprocessing: reshape the image data into rows
    X_train = np.reshape(X_train, (X_train.shape[0], -1))
    X_val = np.reshape(X_val, (X_val.shape[0], -1))
    X_test = np.reshape(X_test, (X_test.shape[0], -1))
    X_dev = np.reshape(X_dev, (X_dev.shape[0], -1))
    

# Normalize the data: subtract the mean image
    mean_image = np.mean(X_train, axis = 0)
    X_train -= mean_image
    X_val -= mean_image
    X_test -= mean_image
    X_dev -= mean_image

-1 表示该位置的数值由reshape的数组实际情况来定


归一化/Normalization

way 1:

X/=np.std(X,axis=0)             # X is after mean subtraction(zero centered) 

way 2:

min =-1 ; max=1;


权值初始化/weight initialization

cs231n的提示: 权值不能全初始化为0; 原因:网络结点的输出一样;梯度也一样;导致更新的梯度也一样。。。。

 训练后合理的权值:一半为正,一半为负

权值初始化的方法: symmetry breaking

W=0.01*np.random.randn(D,H)

randn 取样均值为0,标准差为1的高斯函数

warning: 并不是数字越小越好;梯度与权值成比例,如果权值小,梯度也会小,梯度小会diminish“梯度信号”在BP网络中的流动,对于深度网络来说是一个concern;

你可能感兴趣的:(深度学习,Python)