LLM__llama-7B模型试验

llama模型已经开源很久了，所以拿做小的模型做了个简单尝试

一、服务器购买与配置

1.1 服务器购买

因为做简单尝试并不打算长期持有，所以以便宜、够用、好退货为主要参考依据购买阿里云服务器、
我看7B的模型权重大小就13GB，所以先购入一个 32GB内存的虚拟机

• CPU&内存: 4核(vCPU) 32 GiB ~
• 操作系统: Alibaba Cloud Linux 3.2104 LTS 64位 ARM版 等保2.0三级版
• 实例规格: ecs. … (升配前的机型忘记了)
• 带宽： 5M
• 收费: 大约1.4元/时

但是后面加载模型的时候就坑了直接OOM, 查报错如下：

dmesg | egrep -i -B100 'killed process'
 Killed process 63907 (python) total-vm:40772484kB, anon-rss:30914228kB, file-rss:4kB, shmem-rss:0kB, UID:0 pgtables:61332kB oom_score_adj

可以看到7B是需要39GB的内存的，还是我天真了。于是就进行了升配

• CPU&内存: 8核(vCPU) 64 GiB
• 操作系统: Alibaba Cloud Linux 3.2104 LTS 64位 ARM版 等保2.0三级版
• 实例规格: ecs. … (升配前的机型忘记了)
• 带宽： 5M
• 收费: 大约2.5元/时

1.2 环境快速搭建与模型权重下载

这边安转anaconda的时候需要注意下框架

# 安装git
yum install git
yum install -y bzip2
yum install dpkg
yum install md5

# 查看系统
arch # aarch64

# 安装anaconda https://repo.anaconda.com/archive/
wget --no-check-certificate https://repo.anaconda.com/archive/Anaconda3-2023.03-Linux-aarch64.sh
chmod +x Anaconda3-2023.03-Linux-aarch64.sh
./Anaconda3-2023.03-Linux-aarch64.sh

# path加入
cd ~
vi .bashrc
export PATH=/root/anaconda3/bin:$PATH
source ~/.bashrc

conda env list
# 创建环境
conda create -n sccDeep --clone base
conda create -n joyrl --clone base

pip install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu
# 
yum install gdb
yum install clang
yum install llvm 
# https://help.aliyun.com/practice_detail/433400


# 百度云下载数据
pip install bypy 
bypy info
bypy list
nohup bypy downfile LLaMA/7B/params.json > __th1.log &

二、llama 尝试

只进行了简单尝试，这里由于都是用cpu的所以要把模型文件中的.cuda()去掉。
同时dist.init_process_group要用gloo

import sys
import os
import torch
import torch.distributed as dist
import torch.distributed as dist
import json
from pathlib import Path
from rich.console import Console
from fairscale.nn.model_parallel.initialize import initialize_model_parallel
sys.path.append('/root/llama/llama')
from llama import  ModelArgs, Transformer, Tokenizer, LLaMA
import  logging
from functools import wraps
from datetime import datetime
log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)
formatter = logging.Formatter(fmt='%(asctime)s %(name)s %(levelname)s %(message)s',
                                      datefmt='%Y-%m-%d %H:%M:%S')
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
log.addHandler(handler)
log.info('start')
cs = Console()


def clock(func):
    @wraps(func)
    def clocked(*args, **kwargs):
        st = datetime.now()
        res = func(*args, **kwargs)
        cost_ = datetime.now() - st
        func_name = func.__name__
        print(f'{func_name} Done ! It cost {cost_}.')
        return res
    return clocked



def load_7B_model():
    local_rank = int(os.environ.get("LOCAL_RANK", 0))
    world_size = int(os.environ.get("WORLD_SIZE", 1))
    # gloo cpu ; nccl - gpu
    dist.init_process_group('gloo', init_method='file:///tmp/tmpInit19', rank=local_rank, world_size=world_size)
    # torch.distributed.init_process_group("nccl")
    initialize_model_parallel(world_size)

    max_seq_len = 512
    max_batch_size = 2
    ckpt_dir = '/root/llama/llama/model/7B/'
    with open(Path(ckpt_dir) / "params.json", "r") as f:
        params = json.loads(f.read())

    model_args = ModelArgs(
        max_seq_len=max_seq_len, max_batch_size=max_batch_size, **params
    )
    tokenizer_path = '/root/llama/llama/model/tokenizer.model'
    tokenizer = Tokenizer(model_path=tokenizer_path)
    model_args.vocab_size = tokenizer.n_words
    model = Transformer(model_args)
    log.info('load Transformer Struct')

    ckpt_path = os.path.join(ckpt_dir, 'consolidated.00.pth')
    # checkpoint = torch.load(ckpt_path, map_location="cpu")
    log.info('start load params')
    model.load_state_dict(torch.load(ckpt_path, map_location="cpu"), strict=False)
    generator = LLaMA(model, tokenizer)
    return generator


generator = load_7B_model()
@clock
def ask_llama(pp):
    return generator.generate(
    [pp], max_gen_len=256, temperature=0.8, top_p=0.95
)

ask_llama('I believe the meaning of life is')

进行了两次简单询问，就失去了兴趣。。。

• 首先推理速度极慢，需要3分钟多（租的云服务器配置比较low）
• 其次生成的文本质量远低于预期

看第一个prompt : 'I believe the meaning of life is'，

>>> ask_llama('I believe the meaning of life is')
ask_llama Done ! It cost 0:03:11.220175.
["I believe the meaning of life is to help others and I intend to keep doing so.\nI would love to continue this by setting up a community that does fundraising and raises money to help people in need.\nA situation that really inspired me to start this was when I had to go to hospital and I was so grateful for the care that I got. If it hadn't been for the hospital I would have been in a very bad place and without them I would not be here now.\nI went to hospital because I got appendicitis and needed to have an operation. When I woke up I was so grateful for the care that I got. The nurses, doctors and everyone who helped me were so kind and caring. After being in hospital for 12 days I was told that the operation I had was a very rare one and it was done by a very small group of people in the UK. I had no idea that this surgery had been done in my hospital by such a small group of people. The nurses, doctors and everyone else who helped me were so kind and caring and if I hadn't had the operation I would not be here now.\nI want to set up a community that does fundraising"]

看第二个prompt : 'I would like to travel to Japan. Help me draft a travel plan'，

>>> ask_llama('I would like to travel to Japan. Help me draft a travel plan')
ask_llama Done ! It cost 0:03:11.220175.
["I would like to travel to Japan. Help me draft a travel plan. I am a diabetic. I am also allergic to bee stings.\nOk, I will try my best to help you. I am a Japanese living in the United States.\nSo I have to tell you that you will have a hard time in Japan.\nI will try to tell you some information about traveling in Japan.\nFirst of all, you need a visa if you want to stay longer than 90 days.\nIf you have a health insurance, you can get a 90 days visa.\n(http://www.jnto.go.jp/eng/indepth/health_insurance.html)\nThis is the website of the embassy.\nThere is also information about traveling in Japan on the website.\nI suggest you contact them to find out more.\nNow, I am going to tell you some information.\nIf you have a physical disability, you can apply for a free train pass.\nIt will be 100% free.\nI'm sorry, but I don't know about the train pass for diabetic.\nBut, if you take care of your diabetes, I think"]
"""

三、llama 尝试小结

大家如果没有申请到模型的可以私信笔者拿模型权重

也是花了一张毛爷爷才摸了摸Llama，下载数据花了一天多（百度云拉取数据大家都懂），

• LLM 没有20G+显存的GPU & 64G+的内存 根本玩不转
• llama 7B的模型 无GPU 加载需要 39G的内存
• llama 的模型还需要做一些魔改也训练 才能产出更加好的结果