from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
sequence = "A Titan RTX has 24GB of VRAM"
tokenized_sequence = tokenizer.tokenize(sequence)
print(tokenized_sequence)
报错位置:
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
解决办法:
将双引号改为单引号。