使用happytransformer对gpt-neo进行训练的过程记录

1、环境准备

本文使用的是Python3.10.12的Docker环境

在Python3.10环境中安装以下工具包：xformers、transformers v4.31.0、torch 2.0.1+cu118、happytransformer v2.4.1、accelerate v0.21.0

2、编写python文件

from happytransformer import HappyGeneration, GENSettings, GENTrainArgs
happy_gen = HappyGeneration("GPT-NEO", "EleutherAI/gpt-neo-125m")
args = GENTrainArgs(num_train_epochs=1)
happy_gen.train("/home/data/train.txt", args=args)

That's all.