huggingface pipeline使用模型THUDM/chatglm3-6b

以下代码成功运在CPU机器上:
第一次运行会自动下载模型文件,需要比较长的时间。

from transformers import AutoTokenizer, AutoModel
from transformers import AutoModelForSeq2SeqLM
from huggingface_hub.hf_api import HfFolder

HfFolder.save_token('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')
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True, device='cpu')
model = model.eval()
response = model.chat(tokenizer, "你好", history=[])
print(response)

你可能感兴趣的:(huggingface,java,python,前端)