李宏毅2021课程-机器学习作业2phoneme classification

前言：并没有过strong baseline，public score和provate score为0.705。发现网上对hw2的讨论比较少，所以和大家分享一下。

主要修改部分，

1.将助教的样本代码中的segmiod改成了relu

2.batch size改成了16，epoch改成50

3.采用了L2正则化

4.学习率每5代变成原来的0.1倍

可以修改的地方：修改网络层数，尝试L1正则化，附助教给的思路。pytorch中的model.train自带dropout。

 后附代码：

# fix random seed for reproducibility
same_seeds(0)

# get device 
device = get_device()
print(f'DEVICE: {device}')

# training parameters
num_epoch = 50               # number of training epoch
learning_rate = 0.0001       # learning rate

# the path where checkpoint saved
model_path = './model.ckpt'

# create model, define a loss function, and optimizer
model = Classifier().to(device)
criterion = nn.CrossEntropyLoss() 
#weight_p, bias_p = [],[]
#for name, p in model.named_parameters():
#  if 'bias' in name:
#     bias_p += [p]
3  else:
#     weight_p += [p]
# 这里的model中每个参数的名字都是系统自动命名的，只要是权值都是带有weight，偏置都带有bias，
# 因此可以通过名字判断属性，这个和tensorflow不同，tensorflow是可以用户自己定义名字的，当然也会系统自己定义
#optimizer = torch.optim.Adam([
#          {'params': weight_p, 'weight_decay':0.0001},
#          {'params': bias_p, 'weight_decay':0}
#          ], lr=learning_rate)

weight_decay_list = (param for name, param in model.named_parameters() if name[-4:] != 'bias' and "bn" not in name)
no_decay_list = (param for name, param in model.named_parameters() if name[-4:] == 'bias' or "bn" in name)
parameters = [{'params': weight_decay_list},
              {'params': no_decay_list, 'weight_decay': 0.}]

optimizer = torch.optim.Adam(parameters, lr=learning_rate, weight_decay=5e-4)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.1)#每5代学习率变成原来的0.1