Compile ChatGLM in docker with Nvidia graphic card

At first install docker with nvidia graphic card.

1.create docker with nvidia graphic card

$ sudo docker run -it --gpus all --name chatglm-1  rocker/cuda /bin/bash

If docker has created, start and exec docker:

$ sudo docker start chatglm-1

$ sudo docker exec -it chatglm-1 /bin/bash

2.install the tools

$ apt update

$ apt install sudo wget curl git cmake vim -y

$ apt install software-properties-common

3.install python3.11

$  sudo add-apt-repository ppa:deadsnakes/ppa

$  sudo apt-get update

$  sudo apt-get install python3.11

$  python3.11 --version

4.install anaconda

$ wget https://repo.anaconda.com/archive/Anaconda3-2023.09-0-Linux-x86_64.sh

$ sh ./Anaconda3-2023.09-0-Linux-x86_64.sh

# add the conda path to PATH at the end of ~/.bashrc

$ vim ~/.bashrc 

 export PATH=${PATH}:/your/conda/path

$source ~/.bashrc

5.download the ChatGLM

$ git clone https://github.com/THUDM/ChatGLM2-6B.git

6.download the model from TSINGHUA cloud

清华大学云盘https://cloud.tsinghua.edu.cn/d/674208019e314311ab5c/

Choose the proper model according memory of your graphic  .

6MB:

清华大学云盘https://cloud.tsinghua.edu.cn/d/674208019e314311ab5c/?p=%2Fchatglm2-6b-int4&mode=list

Click every file one by one to download

Such as :

7.add 3 libs 

$ cd ChatGLM2-6B

$ vim ./requirements.txt

…

chardet

streamlit

streamlit-chat

#such as:

8.edit the model path

$ vim web_demo2.py

#use your own model path 

9.create env

$ conda create ChatGLM

$ source activate ChatGLM

10.install the libs for ChatGLM in env

$ pip install -r requirements.txt

#after long time

11.install cuda toolkit

$ sudo apt install nvidia-cuda-toolkit

12.run chatglm

$ python3 -m streamlit run ./web_demo2.py --server.port 8888

13.access and test: http://yourip:8888