ubuntu20安装anaconda tensorflow

Ubuntu20.04安装Pytorch：Anaconda3+Pycharm+Pytorch+GPU_咸咸咸虾的博客-CSDN博客

从源码构建pytorch需要python 3.7以上，安装tensorflow-gpu

conda remove -n python27 --all

conda install astunparse numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses

conda install -c pytorch magma-cuda112

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
# if you are updating an existing checkout
git submodule sync
git submodule update --init --recursive --jobs 0

export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install

安装cuda11.2 cudnn8.1 添加阿里源 安装tensorflow2.6 编译pytorch OpenCV

conda create -n 环境的名字 python=版本号
conda create -n tensorflow2 python=3.7

先安装nvidia dirver

重启

再安装cudatoolkit

不再选择安装cudatoolkit自带的nvidia驱动

Please make sure that
 -   PATH includes /usr/local/cuda-11.2/bin
 -   LD_LIBRARY_PATH includes /usr/local/cuda-11.2/lib64, or, add /usr/local/cuda-11.2/lib64 to /etc/ld.so.conf and run ldconfig as root

To uninstall the CUDA Toolkit, run cuda-uninstaller in /usr/local/cuda-11.2/bin

下载

Download cuDNN v8.1.0 (January 26th, 2021), for CUDA 11.0,11.1 and 11.2

ubuntu安装cuda-11.x会自动生成文件/usr/local/cuda/bin，所以，只需要将cuda路径添加进环境变量就好，同时，要配对cudnn 8.1到相应的文件夹中

sudo cp lib64/* /usr/local/cuda-11.2/lib64/    将lib64下的所有文件拷贝到新地方，注意检查是否拷贝过去的文件是0kb

只需要拷贝到cuda-11.2中，这时lib64和include都在/usr/local/cuda/lib64自动产生相应的文件

gedit ~/.bashrc

设置cuda环境变量的地方

 按照这个阿里云的镜像，安装了tensorflow-gpu-2.5但是不能检测gpu

ubuntu20.04/RTX3090/TensorFlow/pytorch/GPU深度学习环境超详细配置实测

使用pip安装tf-nighly前，可以修改一下pip源为阿里云，以便提升下载速度

~$ sudo mkdir ~/.pip ##创建.pip目录

~$ sudo vim ~/.pip/pip.conf ##新建pip.conf配置文件

文件内容：

[global]

index-url =https://mirrors.aliyun.com/pypi/simple

[install]

trusted-host=mirrors.aliyun.com

首先conda create创建一个dl的环境：

~$ conda create-n dl python=3.8

~$ pip install tf-nightly-gpu ##安装TensorFlow

谨慎使用，平台搭建好后就别再用下面命令更新了

sudo apt update

sudo apt upgrade

sudo apt install build-essential

报错，提示缺少GPU library

https://www.tensorflow.org/install/gpu

加入阿里源之后，使用

pip install tensorflow-gpu==2.6.0

安装成功，测试gpu成功

ERROR: launchpadlib 1.10.13 requires testresources, which is not installed.
ERROR: onnx-graphsurgeon 0.3.10 requires onnx, which is not installed.
Installing collected packages: typing-extensions, six, keras, wheel, numpy, h5py, tensorflow-estimator, absl-py, google-pasta, protobuf, astunparse, keras-preprocessing, gast, wrapt, requests-oauthlib, cachetools, pyasn1, rsa, pyasn1-modules, google-auth, google-auth-oauthlib, zipp, importlib-metadata, markdown, tensorboard-plugin-wit, werkzeug, tensorboard-data-server, grpcio, tensorboard, flatbuffers, opt-einsum, termcolor, clang, tensorflow-gpu
  Attempting uninstall: six
    Found existing installation: six 1.14.0
    Not uninstalling six at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'six'. No files were found to uninstall.
  Attempting uninstall: wheel
    Found existing installation: wheel 0.34.2
    Not uninstalling wheel at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'wheel'. No files were found to uninstall.
  Attempting uninstall: numpy
    Found existing installation: numpy 1.21.2
ERROR: Cannot remove entries from nonexistent file /home/qiao/Downloads/pycuda-2021.1/.eggs/easy-install.pth

  The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.
    
    This package can be installed as:
    ```
    $ pip install nvidia-pyindex
    $ pip install onnx-graphsurgeon
    ```

ERROR: Cannot remove entries from nonexistent file /home/qiao/Downloads/pycuda-2021.1/.eggs/easy-install.pth

手动删除后，重新安装，不行。 最后是手动删除 了这个pycuda-2021.1的文件夹才行的。

download文件中的东西不能随便删除

尝试：I'm assuming you're having an issue with pip install or something that uses it like tox? This is a known issue with anaconda. You can fix the problem by adding the --ignore-installed option when installing your code.

首先安装tensorflow-gpu，然后再安装numpy之类的

ERROR: Cannot uninstall 'certifi'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

pip install certifi --ignore-installed

终于搞定，环境创建后，首先安装TensorFlow-gpu，然后再说其它的

Cannot uninstall ‘certifi‘. It is a distutils installed project_深度检测的博客-CSDN博客

是否需要将这三个路径加入环境变量

Add 3 new paths shown in below figure 7 if they are not already present there. Click on the “New” and enter the below paths. Note that these path may differ for you based on where you have installed the the CUDA and copied the cuDNN folders. To be sure, open the folder where you copied the cuDNN folders and make sure that the below path folders exist there.

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\libnvvp

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\extras\CUPTI\lib64

一定要用sudo安装anaconda

Anaconda3 will now be installed into this location:
/home/qiao/anaconda3

最后一步记得选择同意conda initialize

但是conda create不了新环境

删掉anaconda3这个文件夹，重新用root安装

PREFIX=/root/anaconda3

# To activate this environment, use
#
#     $ conda activate tensorflow25-gpu-36
#
# To deactivate an active environment, use
#
#     $ conda deactivate

安装tensorflow-gpu

Then finally ‘pip install tensorflow-gpu’. 尝试两次，都timeout，添加源后，使用conda install tensorflow-gpu

报错，timeout，

添加源

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
conda config --set show_channel_urls yes

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
conda config --set show_channel_urls yes 

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/

conda update -n base -c defaults conda

conda install -c anaconda tensorflow-gpu 这样装的tensorflow-gpu版本是1.3太低了

成功

 tf.__version__
'1.3.0'

pip uninstall tensorflow

卸载重新安装指定版本的TensorFlow

>>> ts.test.is_gpu_available() 成功找到两块显卡

$conda install opencv-python

tensorflow-2.6.0	3.6-3.9	GCC 7.3.1	Bazel 3.7.2	8.1	11.2
tensorflow-2.5.0	3.6-3.9	GCC 7.3.1	Bazel 3.7.2	8.1	11.2
tensorflow-2.4.0	3.6-3.8	GCC 7.3.1	Bazel 3.1.0	8.0	11.0
tensorflow-2.3.0	3.5-3.8	GCC 7.3.1	Bazel 3.1.0	7.6	10.1

which nvcc能查到nvcc所在的环境变量

在安装CUDA 时候会安装3大组件，分别是 NVIDIA 驱动、toolkit 和 samples。NVIDIA 驱动是用来控制 GPU 硬件，toolkit 里面包括nvcc编译器等，samples或者说SDK 里面包括很多样例程序包括查询设备、带宽测试等等。上面说的 CUDA Driver API是依赖于 NVIDIA 驱动 安装的，而CUDA Runtime API 是通过CUDA toolkit 安装的。
nvidia-smi 和 nvcc 结果的版本为何不一致_JasonLiu1919的博客-CSDN博客_nvidia-smi的cuda版本

查看cudnn版本

~$ cat/usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2

*********************************************************************************************************

anaconda源码编译pytorch

pytorch在无anaconda环境下源码编译安装_FXY.jpg的博客-CSDN博客_不用anaconda安装pytorch

ubuntu18.04源码编译pytorch_zhuikefeng的博客-CSDN博客_pytorch源码编译

安装onnx-graphsurgeon

pip install nvidia-pyindex
pip install onnx-graphsurgeon