cs231n assignment1 q5

前言

往期文章

1. cs231n作业 assignment 1 q1 q2 q3
2. cs231 assignment 1 q4

本次作业我们使用更高级别的特征来学习，而不仅仅是由简简单单的像素点作为训练的特征

之前的几个训练，包括我们自己搭的两层神经网络，单纯的softmax 和 svm, 我们最高的准确率也只有50%,而对图像进行特征提取在进行学习，我们的准确率会进一步的提高

Q5:Higher Level Representations: Image Features

Extract Features

这个内容并不需要我们进行填写，但是我觉得有必要说一下

这里我们在对图像进行抽取特征的处理，我们进行学习的材料不在是单纯的图像的像素值，而是进过我们提取的特征，我们使用两个方法来进行图像的特征提取

1. Histogram of Oriented Gradients (HOG) 注重纹理信息，忽略颜色信息的特征提取，一下是newbing 对他的解释
   
   因为我对图形学并不是非常了解，所以我就贴一下别人对他的详细介绍
   hog特征的介绍 https://zhuanlan.zhihu.com/p/627783852

2. color histogram 注重颜色信息，忽略纹理信息
   
   这里是一个比较简单地解释
   color histogram解释 https://blog.csdn.net/u011280600/article/details/80548871

Train SVM on features

题面

基于上一步提取特征之后，现在让我们基于提取过得特征来进行训练，而不再是简简单单对着像素作为特征训练

解析

这里没啥好说的，我们只是换了学习的素材，代码都不用怎么变

代码

# Use the validation set to tune the learning rate and regularization strength

from cs231n.classifiers.linear_classifier import LinearSVM

learning_rates = [1e-9, 1e-8, 1e-7, 1e-4, 1e-3, 1e-2]
regularization_strengths = [1e-2, 1, 3]

results = {}
best_val = -1
best_svm = None

################################################################################
# TODO:                                                                        #
# Use the validation set to set the learning rate and regularization strength. #
# This should be identical to the validation that you did for the SVM; save    #
# the best trained classifer in best_svm. You might also want to play          #
# with different numbers of bins in the color histogram. If you are careful    #
# you should be able to get accuracy of near 0.44 on the validation set.       #
################################################################################
# *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

import itertools

for lr, reg in itertools.product(learning_rates, regularization_strengths):
    # Create SVM and train it 
    svm = LinearSVM()
    svm.train(X_train_feats, y_train, lr, reg, num_iters=1500)

    # Compute training and validation sets accuracies and append to the dictionary
    y_train_pred, y_val_pred = svm.predict(X_train_feats), svm.predict(X_val_feats)
    results[(lr, reg)] = np.mean(y_train == y_train_pred), np.mean(y_val == y_val_pred)

    # Save if validation accuracy is the best
    if results[(lr, reg)][1] > best_val:
        best_val = results[(lr, reg)][1]
        best_svm = svm


# *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

# Print out results.
for lr, reg in sorted(results):
    train_accuracy, val_accuracy = results[(lr, reg)]
    print('lr %e reg %e train accuracy: %f val accuracy: %f' % (
                lr, reg, train_accuracy, val_accuracy))

print('best validation accuracy achieved: %f' % best_val)

输出

虽然说是准确率应该在44%左右，但是因为我把reg减小和learnging rate调大了，准确率更高了。

Neural Network on image features

题面

跟上面那个一样，就是训练神经网络

解析

但是这个笔记本这里写的有点偷懒了，我们还得自己重置变量，leaning_rate 和 reg 还有result这些，还要自己写输出语句，虽然我是从之前的复制的

还有一点要注意reg不能太大，不然效果超级差

代码

from cs231n.classifiers.fc_net import TwoLayerNet
from cs231n.solver import Solver

input_dim = X_train_feats.shape[1]
hidden_dim = 500
num_classes = 10

data = {
    'X_train': X_train_feats,
    'y_train': y_train,
    'X_val': X_val_feats,
    'y_val': y_val,
    'X_test': X_test_feats,
    'y_test': y_test,
}

net = TwoLayerNet(input_dim, hidden_dim, num_classes)
best_net = None

################################################################################
# TODO: Train a two-layer neural network on image features. You may want to    #
# cross-validate various parameters as in previous sections. Store your best   #
# model in the best_net variable.                                              #
################################################################################
# *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

learning_rates = np.linspace(1e-2, 2.75e-2, 4)
regularization_strengths = np.geomspace(1e-6, 1e-4, 3)

results = {}
best_val = -1

import itertools

for lr, reg in itertools.product(learning_rates, regularization_strengths):
    # Create Two Layer Net and train it with Solver
    model = TwoLayerNet(input_dim, hidden_dim, num_classes,reg = reg)
    solver = Solver(model, data, optim_config={'learning_rate': lr}, num_epochs=15, verbose=False)
    solver.train()

    # Compute validation set accuracy and append to the dictionary
    results[(lr, reg)] = solver.best_val_acc

    # Save if validation accuracy is the best
    if results[(lr, reg)] > best_val:
        best_val = results[(lr, reg)]
        best_net = model

# Print out results.
for lr, reg in sorted(results):
    val_accuracy = results[(lr, reg)]
    print('lr %e reg %e val accuracy: %f' % (lr, reg, val_accuracy))
    
print('best validation accuracy achieved during cross-validation: %f' % best_val)

# *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

输出

结语

从这份作业可以看出确实基于特征的学习比基于像素的学习更好一点，但是并没有好的明显，这份作业主要是让人学了一下基本的深度学习代码编写的技巧