CS231N作业1Two-Layer Neural Network

继承上次作业的softmax计算损失，再额外加个隐藏层
1.完成neural_net.py中scores的计算

    z1=X.dot(W1)+b1
    a1=np.maximum(0,z1)
    scores=a1.dot(W2)+b2

2.完成neural_net.py中loss的计算

    scores_max=np.max(scores,axis=1,keepdims=True)
    scores=scores-scores_max
    p=np.exp(scores)/np.sum(np.exp(scores),axis=1,keepdims=True)
    loss=-np.sum(np.log(p[np.arange(N),y]))/N
    loss+=reg*0.5*np.sum(W1*W1)+reg*0.5*np.sum(W2*W2)

3.完成neural_net.py中grads的计算

    dscores=p
    dscores[range(N),y]-=1
    dscores/=N
    grads['W2']=np.dot(a1.T,dscores)
    grads['b2']=np.sum(dscores,axis=0)
    da1=dscores.dot(W2.T)
    da1[a1<=0]=0
    grads['W1']=np.dot(X.T,da1)
    grads['b1']=np.sum(da1,axis=0)
    grads['W2']+=reg*W2
    grads['W1']+=reg*W1

4.补充neural.net.py文件里train函数的缺失代码
第一段

      sample_indice=np.random.choice(np.arange(num_train),batch_size)
      X_batch=X[sample_indice]
      y_batch=y[sample_indice]

第二段

      self.params['W1'] += -learning_rate * grads['W1']
      self.params['b1'] += -learning_rate * grads['b1']
      self.params['W2'] += -learning_rate * grads['W2']
      self.params['b2'] += -learning_rate * grads['b2']

5.补充neural.net.py文件里predict函数的缺失代码

    z1=X.dot(self.params['W1'])+self.params['b1']
    a1=np.maximum(0,z1)
    scores=a1.dot(self.params['W2'])+self.params['b2']
    y_pred=np.argmax(scores,axis=1)

6.超参数

best_val=-1
best_stats=None
learning_rates=[1e-2,1e-3]
regulation_strength=[0.4,0.5,0.6]
results={}
iters=500
for lr in learning_rates:
    for rs in regulation_strength:
        net=TwoLayerNet(input_size,hidden_size,num_classes)
        stats=net.train(X_train,y_train,X_val,y_val,num_iters=iters,batch_size=200,learning_rate=lr,learning_rate_decay=0.95,reg=rs)
        y_train_pred=net.predict(X_train)
        acc_train=np.mean(y_train_pred==y_train)
        y_val_pred=net.predict(X_val)
        acc_val=np.mean(y_val==y_val_pred)
        results[(lr,rs)]=(acc_train,acc_val)
        if best_val<acc_val:
            best_stats=stats
            best_val=acc_val
            best_net=net
for lr, reg in sorted(results):
    train_accuracy, val_accuracy = results[(lr, reg)]
    print('lr %e reg %e train accuracy: %f val accuracy: %f' % (
                lr, reg, train_accuracy, val_accuracy))

print('best validation accuracy achieved during cross-validation: %f' % best_val)