【cs231n】Assignment1总结

cs231n的第一部分主要为knn，svm，softmax以及two_layer_network，作为入门，主要难点在于损失函数构建及梯度求导。

算法	代码	重难点
KNN	cs231n/classifiers/k_nearest_neighbor.py \qquad knn.ipynb	noloop求距离 交叉验证
SVM	cs231n/classifiers/linear_svm.py	求导
softmax	cs231n/classifiers/softmax.py	求导
neural_net	cs231n/classifiers/neural_net.py	求导

KNN(k-Nearest Neighbor)

KNN算法的步骤如下：

1. 训练时，分类器读入训练集并保存
2. 测试时，kNN通过比较所有训练图像，然后传递k个最相似的训练实例的标签，哪类标签最多，测试图像就属于此类。

其中，求两图像之间的相似度有L1和L2两种算法：$$d_1\left(I_1,I_2\right)=\sum _p\left|I_1^p-I_2^p\right|d_2\left(I_1,I_2\right)=\sqrt{\sum _p{\left(I_1^p-I_2^p\right)}^2}$$

• 求两图像之间的距离

def compute_distances_no_loops(self, X):
    """
    Compute the distance between each test point in X and each training point
    in self.X_train using no explicit loops.
    Input / Output: Same as compute_distances_two_loops
    """
    num_test = X.shape[0]
    num_train = self.X_train.shape[0]
    dists = np.zeros((num_test, num_train)) 
  
    a = np.sum(X*X, axis=1).reshape(num_test,1)  # 500,1
    b = -2 * np.dot(X, self.X_train.T)      # 500,5000
    c = np.sum(self.X_train**2,axis=1).reshape(1,num_train)  #1，5000
    dists = np.sqrt(a+b+c)
    return dists

• 根据距离求分类结果

 def predict_labels(self, dists, k=1):
    """
    Given a matrix of distances between test points and training points,
    predict a label for each test point.
    Inputs:
    - dists: A numpy array of shape (num_test, num_train) where dists[i, j]
      gives the distance betwen the ith test point and the jth training point.
    Returns:
    - y: A numpy array of shape (num_test,) containing predicted labels for the
      test data, where y[i] is the predicted label for the test point X[i].  
    """
    num_test = dists.shape[0]
    y_pred = np.zeros(num_test)
    for i in range(num_test):
      # A list of length k storing the labels of the k nearest neighbors to the ith test point.
      closest_y = []
      closest_y = np.argsort(dists[i])[:k]
      y_pred[i] = np.argmax(np.bincount(self.y_train[closest_y]))
    return y_pred

后面三种算法都遵循一样的方法：

1. 训练时，通过一系列的计算将样本转化为分类标签，并与真实标签进行计算得到损失函数；对损失函数求导，通过梯度下降不断优化分类器的计算参数，降低损失函数
2. 测试时，使用训练好的分类器对测试集样本进行计算，从而得到测试集的预测分类标签。

SVM

softmax

neural_net