open_clip.create_model_and_transforms 加载本地模型

open_clip.create_model_and_transforms 加载本地模型

代码

model, _, preprocess = open_clip.create_model_and_transforms(
        'ViT-L-14', 
        pretrained='laion/CLIP-ViT-L-14-laion2B-s32B-b82K/pytorch_model.bin'
    )

pretrained 指定路径 laion/CLIP-ViT-L-14-laion2B-s32B-b82K/pytorch_model.bin

报错

    model, _, preprocess = open_clip.create_model_and_transforms(
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 382, in create_model_and_transforms
    model = create_model(
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 288, in create_model
    load_checkpoint(model, checkpoint_path)
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 159, in load_checkpoint
    incompatible_keys = model.load_state_dict(state_dict, strict=strict)
  File "/data/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIP:
        Missing key(s) in state_dict: "positional_embedding", "text_projection", "visual.class_embedding", "visual.positional_embedding", "visual.proj"

解决方法

到 https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K/tree/main 下载 open_clip_pytorch_model.bin
修改代码

model, _, preprocess = open_clip.create_model_and_transforms(
        'ViT-L-14', 
        pretrained='laion/CLIP-ViT-L-14-laion2B-s32B-b82K/open_clip_pytorch_model.bin'
    )

再次执行，正常运行！！

参考

https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K/tree/main