【CS231n】Spring 2020 Assignments - Assignment1 - Features

---

前言

我的作业是在 Google Colaboratory 上完成的，还是挺方便的。

本次作业的主要内容为：使用 方向梯度直方图HOG( Histogram of Oriented Gradients) 和 使用色相通道的颜色直方图 来构成图像的特征，将这些特征作为输入而不是原始的图像。

我遇到的主要问题还是参数范围如何确定。因为本次作业没有给出参数值的提示，所以一开始我是在原来的作业中得出的最佳参数附近调试，然后发现准确率只有10%！这很明显是一个糟糕的参数。仔细一想，确实是这样，**输入都变了，这些超参数怎么可能还在原来的范围呢？！**于是采取了大胆的想法，直接在一个超大的范围观察准确率分布。例如神经网络模型中的学习率 learning_rate ，直接在 (0, 1) 内查看准确率分布，再细分三四次就可以得到比较合适的超参数范围了。

---

Train SVM on features

learning_rate_step_size = 0.25000000e-08
regularization_strength_step_size = 0.5e5

best_lr = 0.
best_regularization_strength = 0.

lr = 6.000000e-08
for i in range(20):
  regularization_strength = 2e5
  for j in range(10):
    new_svm = LinearSVM()
    new_svm.train(X_train_feats, y_train, learning_rate=lr, reg=regularization_strength, num_iters=1200, verbose=False)
    y_train_pred = new_svm.predict(X_train_feats)
    training_accuracy = np.mean(y_train == y_train_pred)
    y_val_pred = new_svm.predict(X_val_feats)
    validation_accuracy = np.mean(y_val == y_val_pred)
    if validation_accuracy > best_val:
      best_val = validation_accuracy
      best_svm = new_svm
      best_lr = lr
      best_regularization_strength = regularization_strength
    results[(lr, regularization_strength)] = (training_accuracy, validation_accuracy)
    regularization_strength += regularization_strength_step_size
  lr += learning_rate_step_size

Neural Network on image features

import time

best_acc = 0.0
best_lr = 0.0
best_bs = 0
best_r = 0.0

num_expriment = 3
lr = 0.83  # 0.83~0.86, 0.84 is best
r = 1.25e-7  # 1.25e-7~1.55e-7, 1.3e-7 is best
bs = 190    # 190~250, 210 is best

lr_rate = (0.86 - 0.83) / num_expriment
r_rate = (1.55e-7 - 1.25e-7) / num_expriment
bs_rate = (int)((250 - 190) / num_expriment)

i, total_num = 0, (num_expriment + 1) ** 3
for bsi in range(num_expriment + 1):
  for lri in range(num_expriment + 1):
    for ri in range(num_expriment + 1):
      i += 1
      start_time = time.time()
      net = TwoLayerNet(input_dim, hidden_dim, num_classes)

      # Train the network
      stats = net.train(X_train_feats, y_train, X_val_feats, y_val,
                  num_iters=3000, batch_size=(bs + bsi * bs_rate),
                  learning_rate=(lr + lri * lr_rate), learning_rate_decay=0.95,
                  reg=(r + ri * r_rate), verbose=False)

      # Predict on the validation set
      val_acc = (net.predict(X_val_feats) == y_val).mean()

      print('lr: ', (lr + lri * lr_rate), 
            '; r: ', (r + ri * r_rate), 
            '; bs: ', (bs + bsi * bs_rate), 
            '; VA: ', val_acc, 
            '; Left time: ', (total_num - i) * (time.time() - start_time), 's')

      if val_acc > best_acc:
        best_acc = val_acc
        best_net = net
        best_bs = bs + bsi * bs_rate
        best_lr = lr + lri * lr_rate
        best_r = r + ri * r_rate
        if best_acc >= 0.60:
          break