大模型学习与实践笔记(十四)

使用 OpenCompass 评测 InternLM2-Chat-7B 模型使用 LMDeploy 0.2.0 部署后在 C-Eval 数据集上的性能

步骤1:下载internLM2-Chat-7B 模型,并进行挂载

以下命令将internlm2-7b模型挂载到当前目录下:

ln -s /share/model_repos/internlm2-7b/ ./

大模型学习与实践笔记(十四)_第1张图片

步骤2:编译安装LMdeploy0.2.0

pip install 'lmdeploy[all]==v0.2.0'

步骤3:使用LMdeploy 将模型internLM2-Chat-7B  进行转换

lmdeploy convert internlm2-chat-7b /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b

运行日志:

(internlm-demo) root@intern-studio:~/deploy# lmdeploy convert internlm2-chat-7b /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b
create workspace in directory workspace
copy triton model templates from "/root/.conda/envs/internlm-demo/lib/python3.10/site-packages/lmdeploy/serve/turbomind/triton_models" to "workspace/triton_models"
copy service_docker_up.sh from "/root/.conda/envs/internlm-demo/lib/python3.10/site-packages/lmdeploy/serve/turbomind/service_docker_up.sh" to "workspace"
model_name             internlm2-chat-7b
model_format           None
inferred_model_format  internlm2
model_path             /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b
tokenizer_path         /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b/tokenizer.model
output_format          fp16
01/29 17:36:32 - lmdeploy - WARNING - Can not find tokenizer.json. It may take long time to initialize the tokenizer.
*** splitting layers.0.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.0.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.0.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.0.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.0.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.1.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.1.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.1.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.1.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.1.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.2.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.2.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.2.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.2.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.2.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.3.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.3.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.3.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.3.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.3.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.4.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.4.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.4.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.4.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.4.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.5.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.5.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.5.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.5.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.5.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.6.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.6.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.6.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.6.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.6.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.7.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.7.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.7.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.7.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.7.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.8.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.8.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.8.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.8.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.8.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.9.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.9.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.9.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.9.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.9.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.10.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.10.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.10.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.10.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.10.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.11.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.11.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.11.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.11.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.11.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.12.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.12.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.12.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.12.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.12.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.13.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.13.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.13.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.13.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.13.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.14.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.14.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.14.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.14.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.14.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.15.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.15.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.15.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.15.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.15.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.16.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.16.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.16.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.16.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.16.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.17.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.17.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.17.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.17.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.17.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.18.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.18.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.18.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.18.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.18.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.19.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.19.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.19.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.19.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.19.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.20.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.20.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.20.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.20.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.20.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.21.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.21.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.21.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.21.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.21.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.22.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.22.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.22.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.22.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.22.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.23.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.23.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.23.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.23.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.23.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.24.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.24.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.24.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.24.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.24.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.25.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.25.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.25.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.25.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.25.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.26.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.26.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.26.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.26.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.26.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.27.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.27.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.27.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.27.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.27.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.28.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.28.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.28.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.28.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.28.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.29.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.29.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.29.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.29.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.29.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.30.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.30.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.30.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.30.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.30.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.31.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.31.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.31.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.31.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.31.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
Convert to turbomind format: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:27<00:00,  1.18it/s

步骤4:模型结果测评

首先新建config文件,其中参数”/root/deploy/workspace/“表示LMdeploy转换后的模型地址。

from mmengine.config import read_base
from opencompass.models.turbomind import TurboMindModel

with read_base():
 # choose a list of datasets   
 from .datasets.ceval.ceval_gen_5f30c7 import ceval_datasets 
 # and output the results in a choosen format
 from .summarizers.medium import summarizer

datasets = sum((v for k, v in locals().items() if k.endswith('_datasets')), [])

internlm_meta_template = dict(round=[
 dict(role='HUMAN', begin='<|User|>:', end='\n'),
 dict(role='BOT', begin='<|Bot|>:', end='\n', generate=True),
],
 eos_token_id=103028)

# config for internlm-chat-7b
internlm2_chat_7b = dict(
 type=TurboMindModel,
 abbr='internlm2-chat-7b-turbomind',
 path='/root/deploy/workspace/',
 engine_config=dict(session_len=512,
 max_batch_size=2,
 rope_scaling_factor=1.0),
 gen_config=dict(top_k=1,
 top_p=0.8,
 temperature=1.0,
 max_new_tokens=100),
 max_out_len=100,
 max_seq_len=512,
 batch_size=2,
 concurrency=1,
 meta_template=internlm_meta_template,
 run_cfg=dict(num_gpus=1, num_procs=1),
)
models = [internlm2_chat_7b]

在opencompass 目录下运行:

python run.py configs/eval_turbomind.py

同样可以添加--debug ,输出日志信息。

python run.py configs/eval_turbomind.py --debug

过程日志如下:

大模型学习与实践笔记(十四)_第2张图片

大模型学习与实践笔记(十四)_第3张图片

大模型学习与实践笔记(十四)_第4张图片

你可能感兴趣的:(人工智能,langchain,AIGC,llama,深度学习)