Ubuntu 22.04 x86_64 源码编译 pytorch-v2.0.1 笔记
依赖
cmake
sudo apt install cmakecuda
Ubuntu 20.04 LTS 安装 nvidia 驱动 + cuda 11.8 从开始到放弃！_hkNaruto的博客-CSDN博客
源码
地址
GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration
https://github.com/pytorch/pytorch.git
git archive --format=tar --prefix=pytorch-v2.0.1/ v2.0.1 > /tmp/pytorch-v2.0.1.tar
tar -xvf pytorch-v2.0.1.tar
cd pytorch-v2.0.1
make
故障处理
No CMAKE_CUDA_COMPILER could be found.
-- /usr/bin/c++ /tmp/pytorch-v2.0.1/torch/abi-check.cpp -o /tmp/pytorch-v2.0.1/build/abi-check
-- Determined _GLIBCXX_USE_CXX11_ABI=1
-- Could not find ccache. Consider installing ccache to speed up compilation.
-- Turning off deprecation warning due to glog.
-- Current compiler supports avx2 extension. Will build perfkernels.
-- Current compiler supports avx512f extension. Will build fbgemm.
-- The CUDA compiler identification is unknown
CMake Error at cmake/public/cuda.cmake:47 (enable_language):
No CMAKE_CUDA_COMPILER could be found.
Tell CMake where to find the compiler by setting either the environment
variable "CUDACXX" or the CMake cache entry CMAKE_CUDA_COMPILER to the full
path to the compiler, or to the compiler name if it is in the PATH.
Call Stack (most recent call first):
cmake/Dependencies.cmake:43 (include)
CMakeLists.txt:717 (include)
-- Configuring incomplete, errors occurred!
See also "/tmp/pytorch-v2.0.1/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/pytorch-v2.0.1/build/CMakeFiles/CMakeError.log".
make: *** [Makefile:6：all] 错误 1
解决
export PATH=$PATH:/usr/local/cuda/bin/does not comtain a CMakeLists.txt file
CMake Error at cmake/Dependencies.cmake:434 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/pthreadpool
does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:437 (set_property):
set_property could not find TARGET pthreadpool. Perhaps it has not yet
been created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:477 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/cpuinfo
does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:482 (set_property):
set_property could not find TARGET cpuinfo. Perhaps it has not yet been
created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:504 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/QNNPACK
does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:510 (target_compile_options):
Cannot specify compile options for target "qnnpack" which is not built by
this project.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:515 (set_property):
set_property could not find TARGET qnnpack. Perhaps it has not yet been
created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:516 (set_property):
set_property could not find TARGET cpuinfo. Perhaps it has not yet been
created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:330 (add_subdirectory):
add_subdirectory given source
"/tmp/pytorch-v2.0.1/third_party/cpuinfo/deps/clog" which is not an
existing directory.
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:334 (set_property):
set_property could not find TARGET clog. Perhaps it has not yet been
created.
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:352 (add_subdirectory):
The binary directory
/tmp/pytorch-v2.0.1/build/confu-deps/cpuinfo
is already used to build a source directory. It cannot be used to build
source directory
/tmp/pytorch-v2.0.1/third_party/cpuinfo
Specify a unique binary directory name.
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:362 (add_subdirectory):
The binary directory
/tmp/pytorch-v2.0.1/build/confu-deps/pthreadpool
is already used to build a source directory. It cannot be used to build
source directory
/tmp/pytorch-v2.0.1/third_party/pthreadpool
Specify a unique binary directory name.
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:388 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/FXdiv
does not contain a CMakeLists.txt file.
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:403 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/psimd
does not contain a CMakeLists.txt file.
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:420 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/FP16
does not contain a CMakeLists.txt file.
CMake Error at cmake/Dependencies.cmake:571 (set_property):
set_property could not find TARGET cpuinfo. Perhaps it has not yet been
created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
-- Brace yourself, we are building NNPACK
CMake Error at cmake/External/nnpack.cmake:71 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/NNPACK
does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
cmake/Dependencies.cmake:601 (include)
CMakeLists.txt:717 (include)
CMake Error at cmake/External/nnpack.cmake:76 (set_property):
set_property could not find TARGET nnpack. Perhaps it has not yet been
created.
Call Stack (most recent call first):
cmake/Dependencies.cmake:601 (include)
CMakeLists.txt:717 (include)
CMake Error at cmake/External/nnpack.cmake:77 (set_property):
set_property could not find TARGET pthreadpool. Perhaps it has not yet
been created.
Call Stack (most recent call first):
cmake/Dependencies.cmake:601 (include)
CMakeLists.txt:717 (include)
CMake Error at cmake/External/nnpack.cmake:78 (set_property):
set_property could not find TARGET cpuinfo. Perhaps it has not yet been
created.
Call Stack (most recent call first):
cmake/Dependencies.cmake:601 (include)
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:642 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/XNNPACK
does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:832 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/fbgemm
does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:833 (set_property):
set_property could not find TARGET fbgemm_generic. Perhaps it has not yet
been created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:834 (set_property):
set_property could not find TARGET fbgemm_avx2. Perhaps it has not yet
been created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:835 (set_property):
set_property could not find TARGET fbgemm_avx512. Perhaps it has not yet
been created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:836 (set_property):
set_property could not find TARGET fbgemm. Perhaps it has not yet been
created.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
-- Could NOT find Numa (missing: Numa_INCLUDE_DIR Numa_LIBRARIES)
CMake Warning at cmake/Dependencies.cmake:902 (message):
Not compiling with NUMA. Suppress this warning with -DUSE_NUMA=OFF
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Warning at cmake/Dependencies.cmake:981 (message):
Not compiling with ITT. Suppress this warning with -DUSE_ITT=OFF
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
CMake Error at cmake/public/utils.cmake:188 (set):
set given invalid arguments for CACHE mode.
Call Stack (most recent call first):
cmake/Dependencies.cmake:983 (caffe2_update_option)
CMakeLists.txt:717 (include)
CMake Error at cmake/Dependencies.cmake:1000 (add_subdirectory):
The source directory
/tmp/pytorch-v2.0.1/third_party/FP16
does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
-- Using third party subdirectory Eigen.
-- Setting Python's library to /usr/lib/python3.10
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.10.12", minimum required is "3.0")
-- Found PythonLibs: /usr/lib/python3.10 (Required is at least version "3.0")
CMake Error at cmake/Dependencies.cmake:1088 (if):
if given arguments:
"VERSION_LESS" "3"
Unknown arguments specified
Call Stack (most recent call first):
CMakeLists.txt:717 (include)
-- Configuring incomplete, errors occurred!
See also "/tmp/pytorch-v2.0.1/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/pytorch-v2.0.1/build/CMakeFiles/CMakeError.log".
make: *** [Makefile:6：all] 错误 1
切换到git 源码目录重试
cd ~/Downloads/src/pytorch/
git checkout v2.0.1
git submodule init
git submodule update注意，调整下eigen地址
[submodule "third_party/eigen"]
ignore = dirty
path = third_party/eigen
#url = https://gitlab.com/libeigen/eigen.git
url = https://gitee.com/masa-laboratory/eigen
根据以往stable diffusion webui经验，采用conda创建python3.10.6
conda create -n pytorch-build python=3.10.6
conda activate pytorch-buildNo module named 'yaml'
ModuleNotFoundError: No module named 'yaml'
CMake Error at cmake/Codegen.cmake:197 (message):
Failed to get generated_headers list
Call Stack (most recent call first):
caffe2/CMakeLists.txt:2 (include)
解决
pip install pyyamltarget specific option mismatch
error: inlining failed in call to ‘always_inline’ ‘_mm512_set_epi8’: target specific option mismatch
121 | _mm512_set_epi8 (char __q63, char __q62, char __q61, char __q60,
| ^~~~~~~~~~~~~~~
估计是当前gcc版本过高
gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04) 待续。。。