【实战】minigpt4的体验和微调

MiniGPT-4

https://github.com/vision-cair/minigpt-4

1 环境配置

1.1 安装环境

git lfs install
//如果报错 :git: 'lfs' is not a git command. See 'git --help'.
//尝试使用:
sudo apt-get install git-lfs
git lfs install

1.2 准备Vicuna权重

(1)下载 Vicuna’s delta weight

git lfs install
git clone https://huggingface.co/lmsys/vicuna-13b-delta-v0  # more powerful, need at least 24G gpu memory
# or
git clone https://huggingface.co/lmsys/vicuna-7b-delta-v0  # smaller, need 12G gpu memory

(2)填写表格以获得原始的LLAMA-7B或LLAMA-13B权重
这个东西其实也不用这么老实,直接huggingface上拉下来就行
(3)下载兼容library

pip install git+https://github.com/lm-sys/[email protected]

(4)创建最终权重

python -m fastchat.model.apply_delta --base /path/to/llama-13bOR7b-hf/  --target /path/to/save/working/vicuna/weight/  --delta /path/to/vicuna-13bOR7b-delta-v0/

同时修改minigpt4/configs/models/minigpt4.yaml中llama_model的路径
报错:

Tokenizer class LLaMATokenizer does not exist or is not currently imported.

参考#59将llama-13b-hf/tokenizer_config.json 中的"tokenizer_class": “LLaMATokenizer” 改成 “tokenizer_class”: “LlamaTokenizer”

1.3 下载预训练权重

百度网盘: Vicuna 7B:

谷歌:Vicuna 13B
https://drive.google.com/file/d/1a4zLvaiDBr-36pasffmgpvH5P7CKmpze/view
Vicuna 7B:
https://drive.google.com/file/d/1RY9jV0dyqLX-o38LrumkKRh6Jtaop58R/view

同时修改配置文件中路径到实际下载路径

2. 体验

python demo.py --cfg-path eval_configs/minigpt4_eval.yaml  --gpu-id 0

【实战】minigpt4的体验和微调_第1张图片

3.微调

预训练贫困小孩就算了,不过就算是微调至少要16G

train_configs/minigpt4_stage2_finetune.yaml配置

  weight_decay: 0.05
  max_epoch: 5
  iters_per_epoch: 20
  batch_size_train: 1 #12
  batch_size_eval: 1 #12
  num_workers: 2
  warmup_steps: 200

修改两个路径
(1)train_configs/minigpt4_stage2_pretrain.yaml为第一阶段的训练checkpoint
(2minigpt4/configs/datasets/cc_sbu/align.yaml修改为数据集路径

官方有提供训练集和第一阶段预训练模型

训练集格式如下:

{“annotations”: [{“image_id”: “2”, “caption”: “The image shows a man
fishing on a lawn next to a river with a bridge in the background.
Trees can be seen on the other side of the river, and the sky is
cloudy.”}
对应图片:
【实战】minigpt4的体验和微调_第2张图片

NUM_GPU为自己的gpu数量

torchrun --nproc-per-node NUM_GPU train.py --cfg-path train_configs/minigpt4_stage2_finetune.yaml

你可能感兴趣的:(python,开发语言)