Linux上安装python虚拟环境

1. 官网下载想要的python版本Python Source Releases | Python.org

2. 解压

tar -xf Python-3.7.2.tar.xz

3. 给linux安装一些必备包

apt update && apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev wget

4. 安装python

cd Python-3.7.2/
./configure --enable-optimizations --prefix=/usr/local/python/python3.7.2
make altinstall

5. 编辑.bashrc文件

vi .bashrc

alias python3.7='/usr/local/python/python3.7.2/bin/python3.7'
alias pip3.7='/usr/local/python/python3.7.2/bin/pip3.7'

source ~/.bashrc

6. 建立虚拟环境

cd env
python3.7 -m venv fairseq
source fairseq/bin/activate
python --version
pip --version

7. 安装torch

# 注意根据CUDA版本(nvcc -V)安装对应的pytorch,官网https://pytorch.org/get-started/previous-versions/
# python版本也要和torch匹配
pip install torch==1.8.1+cu101 torchvision==0.9.1+cu101 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

安装其他包

1. apex

参考fairseq的README.md的命令

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" \
  --global-option="--deprecated_fused_adam" --global-option="--xentropy" \
  --global-option="--fast_multihead_attn" ./

容易报错，注意事项

要安装git  apt-get install git

python版本3.7及以上

torch的版本一定要和CUDA对应(经测试只有参考步骤7中的torch版本有用，对于cuda10.1)

C，C++编译器要调整到8及以下

# Check the original gcc version by
gcc --version
# My results:
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

# Install gcc-8 and g++-8
sudo apt -y install gcc-8 g++-8

# Let system to manage different version of gcc and g++
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 8
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 9
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-8 8

# Change default gcc to gcc-8
sudo update-alternatives --config gcc
select: 1            /usr/bin/gcc-8   8         manual mode

# Change default g++ to g++-8
sudo update-alternatives --config g++
select: 1            /usr/bin/g++-8   8         manual mode

 2. fastBPE

注意：python版本不能过高，python3.9不行

---

参考连接：

【CUDA】nvcc和nvidia-smi显示的版本不一致？ - 简书 (jianshu.com)