Ubuntu18.04下python3.8.8安装spconv过程与错误解决
官方GitHub地址：https://github.com/traveller59/spconv
本人电脑环境：
lijinxin@Ubuntu18:~$ python
Python 3.8.8 (default, Mar 19 2021, 11:33:36)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
lijinxin@Ubuntu18:~$ nvidia-smi
Wed Jun 2 10:52:00 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.80 Driver Version: 460.80 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 GeForce RTX 3070 Off | 00000000:01:00.0 On | N/A |
| 0% 37C P8 23W / 240W | 235MiB / 7979MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 4837 G /usr/lib/xorg/Xorg 185MiB |
+-----------------------------------------------------------------------------+
以下为官方所述：
Install on Ubuntu 16.04/18.04
- if you are using pytorch 1.4+ and encounter "nvcc fatal: unknown -Wall", you need to go to torch package dir and remove flags contains "-Wall" in INTERFACE_COMPILE_OPTIONS in Caffe2Targets.cmake. This problem can't be fixed in this project (to avoid this, I need to remove all torch dependency in cuda sources and drop half support).
-
Use
git clone xxx.git --recursiveto clone this repo. -
Install boost headers to your system include path, you can use either
sudo apt-get install libboost-all-devor download compressed files from boost official website and copy headers to include path. -
Download cmake >= 3.13.2, then add cmake executables to PATH.
-
Ensure you have installed pytorch 1.0+ in your environment, run
python setup.py bdist_wheel(don't usepython setup.py install). -
Run
cd ./dist, use pip to install generated whl file.
前3步没问题，第4步出现报错。执行python setup.py bdist_wheel错误如下：
lijinxin@Ubuntu18:~/mycode/spconv$ python setup.py bdist_wheel
/usr/local/python3.8/lib/python3.8/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
warnings.warn(
running bdist_wheel
running build
running build_py
running build_ext
Release
|||||CMAKE ARGS||||| ['-DCMAKE_PREFIX_PATH=/home/lijinxin/.local/lib/python3.8/site-packages/torch', '-DPYBIND11_PYTHON_VERSION=3.8', '-DSPCONV_BuildTests=OFF', '-DPYTORCH_VERSION=10801', '-DCMAKE_CUDA_FLAGS="--expt-relaxed-constexpr" -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/lijinxin/mycode/spconv/build/lib.linux-x86_64-3.8/spconv', '-DCMAKE_BUILD_TYPE=Release']
-- Caffe2: CUDA detected: 11.2
-- Caffe2: CUDA nvcc is: /usr/local/cuda-11.2/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.2
-- Caffe2: Header version is: 11.2
-- Found cuDNN: v8.1.1 (include: /usr/local/cuda-11.2/include, library: /usr/local/cuda-11.2/lib64/libcudnn.so)
-- /usr/local/cuda-11.2/lib64/libnvrtc.so shorthash is 6b8662e7
-- Autodetected CUDA architecture(s): 8.6
-- Added CUDA NVCC flags for: -gencode;arch=compute_86,code=sm_86
-- Autodetected CUDA architecture(s): 8.6
-- pybind11 v2.6.3 dev1
CMake Error at /usr/local/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find PythonInterp: Found unsuitable version "3.6.9", but required
is at least "3.8" (found /usr/bin/python3)
Call Stack (most recent call first):
/usr/local/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:592 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/cmake-3.20/Modules/FindPythonInterp.cmake:169 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
third_party/pybind11/tools/FindPythonLibsNew.cmake:98 (find_package)
third_party/pybind11/tools/pybind11Tools.cmake:45 (find_package)
third_party/pybind11/tools/pybind11Common.cmake:205 (include)
third_party/pybind11/CMakeLists.txt:190 (include)
-- Configuring incomplete, errors occurred!
See also "/home/lijinxin/mycode/spconv/build/temp.linux-x86_64-3.8/CMakeFiles/CMakeOutput.log".
See also "/home/lijinxin/mycode/spconv/build/temp.linux-x86_64-3.8/CMakeFiles/CMakeError.log".
retcode: 1
kwargs: {'cwd': 'build/temp.linux-x86_64-3.8', 'env': {'CLUTTER_IM_MODULE': 'xim', 'LD_LIBRARY_PATH': '/usr/local/cuda-11.2/lib64:', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'LC_MEASUREMENT': 'zh_CN.UTF-8', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'LC_PAPER': 'zh_CN.UTF-8', 'LC_MONETARY': 'zh_CN.UTF-8', 'XDG_MENU_PREFIX': 'gnome-', 'LANG': 'en_US.UTF-8', 'DISPLAY': ':0', 'GNOME_SHELL_SESSION_MODE': 'ubuntu', 'COLORTERM': 'truecolor', 'USERNAME': 'lijinxin', 'XDG_VTNR': '2', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'MANDATORY_PATH': '/usr/share/gconf/ubuntu.mandatory.path', 'LC_NAME': 'zh_CN.UTF-8', 'XDG_SESSION_ID': '2', 'USER': 'lijinxin', 'DESKTOP_SESSION': 'ubuntu', 'APOLLO_HOME': '/home/lijinxin', 'QT4_IM_MODULE': 'fcitx', 'TEXTDOMAINDIR': '/usr/share/locale/', 'GNOME_TERMINAL_SCREEN': '/org/gnome/Terminal/screen/369d63dc_2572_417b_a386_13c9a6cd3dcc', 'DEFAULTS_PATH': '/usr/share/gconf/ubuntu.default.path', 'PWD': '/home/lijinxin/mycode/spconv', 'HOME': '/home/lijinxin', 'TEXTDOMAIN': 'im-config', 'SSH_AGENT_PID': '5174', 'QT_ACCESSIBILITY': '1', 'XDG_SESSION_TYPE': 'x11', 'XDG_DATA_DIRS': '/usr/share/ubuntu:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop', 'XDG_SESSION_DESKTOP': 'ubuntu', 'LC_ADDRESS': 'zh_CN.UTF-8', 'GJS_DEBUG_OUTPUT': 'stderr', 'LC_NUMERIC': 'zh_CN.UTF-8', 'GTK_MODULES': 'gail:atk-bridge', 'WINDOWPATH': '2', 'TERM': 'xterm-256color', 'SHELL': '/bin/bash', 'VTE_VERSION': '5202', 'QT_IM_MODULE': 'fcitx', 'XMODIFIERS': '@im=fcitx', 'IM_CONFIG_PHASE': '2', 'XDG_CURRENT_DESKTOP': 'ubuntu:GNOME', 'GPG_AGENT_INFO': '/run/user/1000/gnupg/S.gpg-agent:0:1', 'GNOME_TERMINAL_SERVICE': ':1.82', 'XDG_SEAT': 'seat0', 'SHLVL': '1', 'LC_TELEPHONE': 'zh_CN.UTF-8', 'GDMSESSION': 'ubuntu', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'LOGNAME': 'lijinxin', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'XDG_RUNTIME_DIR': '/run/user/1000', 'XAUTHORITY': '/run/user/1000/gdm/Xauthority', 'XDG_CONFIG_DIRS': '/etc/xdg/xdg-ubuntu:/etc/xdg', 'PATH': '/usr/local/cuda-11.2/bin:/home/lijinxin/.local/bin:/home/lijinxin/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/lijinxin/bin', 'LC_IDENTIFICATION': 'zh_CN.UTF-8', 'GJS_DEBUG_TOPICS': 'JS ERROR;JS LOG', 'SESSION_MANAGER': 'local/Ubuntu18:@/tmp/.ICE-unix/5080,unix/Ubuntu18:/tmp/.ICE-unix/5080', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'GTK_IM_MODULE': 'fcitx', 'LC_TIME': 'zh_CN.UTF-8', 'OLDPWD': '/home/lijinxin/mycode', '_': '/usr/local/python3.8/bin/python3.8', 'PLAT': 'linux-x86_64', 'CXXFLAGS': ' -DVERSION_INFO=\\"1.2.1\\"'}}
cmd: None
Traceback (most recent call last):
File "setup.py", line 96, in
setup(
File "/usr/local/python3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
return distutils.core.setup(**attrs)
File "/usr/local/python3.8/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/python3.8/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/python3.8/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/lijinxin/.local/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/usr/local/python3.8/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/python3.8/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/python3.8/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/python3.8/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/python3.8/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "setup.py", line 48, in run
self.build_extension(ext)
File "setup.py", line 91, in build_extension
subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
File "/usr/local/python3.8/lib/python3.8/subprocess.py", line 366, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/home/lijinxin/mycode/spconv', '-DCMAKE_PREFIX_PATH=/home/lijinxin/.local/lib/python3.8/site-packages/torch', '-DPYBIND11_PYTHON_VERSION=3.8', '-DSPCONV_BuildTests=OFF', '-DPYTORCH_VERSION=10801', '-DCMAKE_CUDA_FLAGS="--expt-relaxed-constexpr" -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/lijinxin/mycode/spconv/build/lib.linux-x86_64-3.8/spconv', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
由上述报错可知是由于python版本导致，我用的是python3.8，在系统中已经更改了软连接，即
python——python3.8.8
python3——python3.6.9
然而在上述代码中有调用python3，如下所示
is at least "3.8" (found /usr/bin/python3)因此，我将python3的软连接也指向python3.8，即执行
sudo rm /usr/bin/python3
sudn ln -s /usr/local/python3.8/bin/python3.8 /usr/bin/python3然后再执行python setup.py bdist_wheel，第4步完美运行！
在执行完毕之后，进入./dist, use pip to install generated whl file.执行如下
cd ./dist
sudo chmod +x spconv-1.2.1-cp38-cp38-linux_x86_64.whl
pip install spconv-1.2.1-cp38-cp38-linux_x86_64.whl但是遇到了下方错误：
lijinxin@Ubuntu18:~/mycode/spconv/dist$ pip install spconv-1.2.1-cp38-cp38-linux_x86_64.whl
Defaulting to user installation because normal site-packages is not writeable
ERROR: Exception:
Traceback (most recent call last):
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
status = self.run(options, args)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 204, in wrapper
return func(self, options, args)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 269, in run
session = self.get_default_session(options)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 77, in get_default_session
self._session = self.enter_context(self._build_session(options))
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 87, in _build_session
session = PipSession(
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/network/session.py", line 275, in __init__
self.headers["User-Agent"] = user_agent()
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/network/session.py", line 132, in user_agent
linux_distribution = distro.linux_distribution() # type: ignore
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 125, in linux_distribution
return _distro.linux_distribution(full_distribution_name)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 681, in linux_distribution
self.version(),
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 741, in version
self.lsb_release_attr('release'),
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 903, in lsb_release_attr
return self._lsb_release_info.get(attribute, '')
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 556, in __get__
ret = obj.__dict__[self._fname] = self._f(obj)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 1014, in _lsb_release_info
stdout = subprocess.check_output(cmd, stderr=devnull)
File "/usr/local/python3.8/lib/python3.8/subprocess.py", line 417, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/local/python3.8/lib/python3.8/subprocess.py", line 518, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('lsb_release', '-a')' returned non-zero exit status 1.
Traceback (most recent call last):
File "/usr/local/python3.8/bin/pip3.8", line 8, in
sys.exit(main())
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 71, in main
return command.main(cmd_args)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 104, in main
return self._main(args)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 221, in _main
self.handle_pip_version_check(options)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 143, in handle_pip_version_check
session = self._build_session(
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 87, in _build_session
session = PipSession(
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/network/session.py", line 275, in __init__
self.headers["User-Agent"] = user_agent()
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_internal/network/session.py", line 132, in user_agent
linux_distribution = distro.linux_distribution() # type: ignore
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 125, in linux_distribution
return _distro.linux_distribution(full_distribution_name)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 681, in linux_distribution
self.version(),
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 741, in version
self.lsb_release_attr('release'),
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 903, in lsb_release_attr
return self._lsb_release_info.get(attribute, '')
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 556, in __get__
ret = obj.__dict__[self._fname] = self._f(obj)
File "/home/lijinxin/.local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 1014, in _lsb_release_info
stdout = subprocess.check_output(cmd, stderr=devnull)
File "/usr/local/python3.8/lib/python3.8/subprocess.py", line 417, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/local/python3.8/lib/python3.8/subprocess.py", line 518, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('lsb_release', '-a')' returned non-zero exit status 1.
分析是缺少了lsb_release依赖，因此在根目录下执行：
lijinxin@Ubuntu18:/usr$ sudo find / -name 'lsb_release.py'
find: ‘/run/user/1000/gvfs’: Permission denied
/usr/share/pyshared/lsb_release.py
/usr/lib/python3/dist-packages/lsb_release.py
/usr/lib/python2.7/dist-packages/lsb_release.py
lijinxin@Ubuntu18:/usr$ sudo cp /usr/lib/python3/dist-packages/lsb_release.py /usr/local/python3.8/lib/python3.8/再次在./dist执行：
lijinxin@Ubuntu18:~/mycode/spconv/dist$ pip install spconv-1.2.1-cp38-cp38-linux_x86_64.whl
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: http://mirrors.aliyun.com/pypi/simple
Processing ./spconv-1.2.1-cp38-cp38-linux_x86_64.whl
Installing collected packages: spconv
Successfully installed spconv-1.2.1
完成安装！