huggingface使用预训练模型入门
huggingface使用预训练模型入门
huggingface的官方文档的地址：https://huggingface.co/docs/transformers/quicktour。在官方文档中提供了两种加载预训练模型的方式：一是pipline，二是为pytorch/tensorflow/flax框架加载的预训练模型，本文档只对pytorch进行介绍，其他架构除了名字使用方法一样。
pipline
官方的pipline是针对任务的，具体有哪些任务如下所示，不做过多介绍，比较简单。
使用方式就是：classfier = pipline(task=task_name, model=model_name)， classfier(inputs)，huggingface提供的
pytorch的预训练模型加载
都是在AutoClass模块下进行加载。分成三种：Generic model classes (无头纯模型)，Generic pretraining classes(模型进行预训练)， NLP/CV/AUDIO/MultiModle。
AutoModel加载无头模型
huggingface文档中加载AutoModel有两种方式：from_config/from_pretrain
from_config
from transformers import AutoConfig, AutoModel
# Download configuration from huggingface.co and cache.
config = AutoConfig.from_pretrained("bert-base-cased")
model = AutoModel.from_config(config)
from_pretrained(model_name or model_path)
from transformers import AutoConfig, AutoModel
# Download model and configuration from huggingface.co and cache.
model = AutoModel.from_pretrained("bert-base-cased")
# Update configuration during loading,也就是说可以传入自己定义的一些关键字，会更新到model_config中
model = AutoModel.from_pretrained("bert-base-cased", output_attentions=True)
model.config.output_attentions
# Loading from a TF checkpoint file instead of a PyTorch model (slower)
config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
model = AutoModel.from_pretrained(
"./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
)
AutoModelForPretraining 加载用于预训练的模型
两种方式：from_config/from_pretrain
from_config
from transformers import AutoConfig, AutoModelForPreTraining
# Download configuration from huggingface.co and cache.
config = AutoConfig.from_pretrained("bert-base-cased")
model = AutoModelForPreTraining.from_config(config)
from_pretrained
from transformers import AutoConfig, AutoModelForPreTraining
# Download model and configuration from huggingface.co and cache.
model = AutoModelForPreTraining.from_pretrained("bert-base-cased")
# Update configuration during loading
model = AutoModelForPreTraining.from_pretrained("bert-base-cased", output_attentions=True)
model.config.output_attentions
# Loading from a TF checkpoint file instead of a PyTorch model (slower) , 可以加载TensorFlow模型
config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
model = AutoModelForPreTraining.from_pretrained(
"./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
)
AutoModel加载带头模型
huggingface提供了很管带有任务头的模型，这里只举个典型例子，加载方式和上述一样，都是from_config/from_pretrain
from_config
from transformers import AutoConfig, AutoModelForMaskedLM
# Download configuration from huggingface.co and cache.
config = AutoConfig.from_pretrained("bert-base-cased")
model = AutoModelForMaskedLM.from_config(config)
from_pretrained
from transformers import AutoConfig, AutoModelForMaskedLM
# Download model and configuration from huggingface.co and cache.
model = AutoModelForMaskedLM.from_pretrained("bert-base-cased")
# Update configuration during loading
model = AutoModelForMaskedLM.from_pretrained("bert-base-cased", output_attentions=True)
model.config.output_attentions
# Loading from a TF checkpoint file instead of a PyTorch model (slower)
config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
model = AutoModelForMaskedLM.from_pretrained(
"./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
)