【ModelScope】KAN-TTS搭建SAMBERT个性化语音合成开发环境
KAN-TTS搭建SAMBERT个性化语音合成开发环境
- 1、下载源码和预训练模型
-
- 1.1 安装Git 和 LFS支持
- 1.2 下载KAN-TTS源码
- 1.3 下载预训练模型
- 2、配置conda安装源
-
- 2.1 查看conda源配置
- 2.2 命令增加源
- 2.3 删除源配置
- 2.4 直接编辑配置文件
- 2.5 独立设置pip命令源
- 3、创建开发环境
-
- 3.1 创建环境maas
- 3.2 遇到pytorch_wavelets 模块报错 GnuTLS recv error (-110)
-
- 3.2.1 方法1-重新编译Git安装OpenSSL版本
-
- 3.2.1.1 安装编译git所需依赖库
- 3.2.1.2 切换到编译目录
- 3.2.1.3 下载git的源码
- 3.2.1.4 修改Gnutls库为OpenSSL库依赖
- 3.2.1.5 编译安装Git
- 3.2.2 方法2-用git单独安装pytorch_wavelets
-
- 3.2.2.1 禁用安装pytorch_wavelets 模块
- 3.2.2.2 手动安装pytorch_wavelets 模块
- 4、错误处理
-
- 4.1 SoX could not be found!
- 4.2 ONNXRuntime
-
- 4.2.1 [ONNXRuntimeError] : 7 : INVALID_PROTOBUF
- 4.2.2 import onnxruntime
- 4.2.3 Since ORT 1.9, you are required to explicitly set the providers parameter when instantiating InferenceSession.
- 4.3 sm_89 is not compatible with the current PyTorch installation.
-
- 4.3.1 重新安装CUDA Toolkit 11.8
- 4.3.2 虚拟环境下重新安装PyTorch匹配cuda 11.7
- 4.3.3 PyTorch支持CUDA版本匹配测试
- 4.4 Could not load library libcudnn_cnn_infer.so.x
-
- 4.4.1 Could not load library libcudnn_cnn_infer.so.8
- 4.4.2 ImportError: libffi.so.7: cannot open shared object file: No such file or directory
- 4.5 多卡训练/分布式训练报错MKL_THREADING_LAYER=INTEL
- 4.6 推理错误
-
- 4.6.1 model_dir没用绝对路径 filelist = os.listdir(ckpt_path) FileNotFoundError: [Errno 2] No such file or directory
1、下载源码和预训练模型
1.1 安装Git 和 LFS支持
安装Git
sudo apt install git
Git LFS（Large File Storage, 大文件存储）
sudo apt-get install git-lfs
git lfs install
curl 16 Error：
error: RPC failed; curl 16 Error in the HTTP2 framing layer
fatal: expected flush after ref listing
- 强制git使用HTTP 1.1
git config --global http.version HTTP/1.1- 想要设置回HTTP2
git config --global http.version HTTP/2
1.2 下载KAN-TTS源码
跳转到项目目录
cd ~/msai
msai是我的ai项目目录
下载源码并跳转到源码目录
git clone https://github.com/alibaba-damo-academy/KAN-TTS.git
cd KAN-TTS
1.3 下载预训练模型
git clone https://www.modelscope.cn/damo/speech_personal_sambert-hifigan_nsf_tts_zh-cn_pretrain_16k.git
2、配置conda安装源
2.1 查看conda源配置
conda config --show-sources
2.2 命令增加源
conda config --add channels https://pypi.tuna.tsinghua.edu.cn/simple
2.3 删除源配置
删除指定源
conda config --remove channels https://pypi.tuna.tsinghua.edu.cn/simple
删除所有自定义源 还原默认源
conda config --remove-key channels
2.4 直接编辑配置文件
sudo vim /home/你的用户名/.condarc
channels:
- defaults
custom_channels:
conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
pytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
default_channels:
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
show_channel_urls: True
清理源索引缓存让新配置生效
conda clean -i
2.5 独立设置pip命令源
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
Writing to /home/fb/.config/pip/pip.conf
3、创建开发环境
3.1 创建环境maas
创建名为maas的python开发环境
conda env create -f environment.yaml
3.2 遇到pytorch_wavelets 模块报错 GnuTLS recv error (-110)
原因是git版本使用了GnuTLS库，不是OpenSSL
Collecting git+https://github.com/fbcotter/pytorch_wavelets.git (from -r /mnt/d/develop/Ubuntu/kan-tts/condaenv.s5y8fuxa.requirements.txt (line 19))
Cloning https://github.com/fbcotter/pytorch_wavelets.git to /tmp/pip-req-build-utwg371g
Pip subprocess error:
Running command git clone --filter=blob:none --quiet https://github.com/fbcotter/pytorch_wavelets.git /tmp/pip-req-build-utwg371g
fatal: unable to access 'https://github.com/fbcotter/pytorch_wavelets.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated.
error: subprocess-exited-with-error
× git clone --filter=blob:none --quiet https://github.com/fbcotter/pytorch_wavelets.git /tmp/pip-req-build-utwg371g did not run successfully.
│ exit code: 128
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× git clone --filter=blob:none --quiet https://github.com/fbcotter/pytorch_wavelets.git /tmp/pip-req-build-utwg371g did not run successfully.
│ exit code: 128
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
failed
CondaEnvException: Pip failed
3.2.1 方法1-重新编译Git安装OpenSSL版本
重新编译Git从GnuTLS版本到OpenSSL版本
3.2.1.1 安装编译git所需依赖库
sudo apt-get install build-essential fakeroot dpkg-dev
sudo apt-get build-dep git
sudo apt-get install libcurl4-openssl-dev
sudo apt install libssl-dev
3.2.1.2 切换到编译目录
cd ~
mkdir src-git
cd src-git
3.2.1.3 下载git的源码
apt-get source git
(base) fb@VP01:~/src-git$ apt-get source git
Reading package lists... Done
E: You must put some 'deb-src' URIs in your sources.list
这是deb-src源码地址被注释掉了，以提高apt update速度，我们需要开启才能下载git的源码
备份源列表
sudo cp /etc/apt/sources.list /etc/apt/sources.list.back
修改源列表，去年deb-src前面的#注释
sudo vim /etc/apt/sources.list
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu/ jammy main restricted
deb-src http://archive.ubuntu.com/ubuntu/ jammy main restricted
## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ jammy universe
deb-src http://archive.ubuntu.com/ubuntu/ jammy universe
deb http://archive.ubuntu.com/ubuntu/ jammy-updates universe
deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates universe
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://archive.ubuntu.com/ubuntu/ jammy multiverse
deb-src http://archive.ubuntu.com/ubuntu/ jammy multiverse
deb http://archive.ubuntu.com/ubuntu/ jammy-updates multiverse
deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates multiverse
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu/ jammy-security main restricted
deb-src http://security.ubuntu.com/ubuntu/ jammy-security main restricted
deb http://security.ubuntu.com/ubuntu/ jammy-security universe
deb-src http://security.ubuntu.com/ubuntu/ jammy-security universe
deb http://security.ubuntu.com/ubuntu/ jammy-security multiverse
deb-src http://security.ubuntu.com/ubuntu/ jammy-security multiverse
更新源列表
sudo apt-get update
重新下载git源码
apt-get source git
3.2.1.4 修改Gnutls库为OpenSSL库依赖
进入git源码目录
cd git-2.34.1
(base) fb@VP01:~/src-git$ ll
total 7240
drwxr-xr-x 3 fb fb 4096 Apr 10 00:11 ./
drwxr-x--- 13 fb fb 4096 Apr 9 21:15 ../
drwxr-xr-x 28 fb fb 20480 Apr 10 00:11 git-2.34.1/
-rw-r--r-- 1 fb fb 753396 Feb 15 02:53 git_2.34.1-1ubuntu1.8.debian.tar.xz
-rw-r--r-- 1 fb fb 2927 Feb 15 02:53 git_2.34.1-1ubuntu1.8.dsc
-rw-r--r-- 1 fb fb 6623760 Jan 25 2022 git_2.34.1.orig.tar.xz
(base) fb@VP01:~/src-git$ cd git-2.34.1
(base) fb@VP01:~/src-git/git-2.34.1$
不知道是 2.x.x，
可以用通配符进入第一个匹配目录
cd git-2.*.*/也可以用ll列出详细目录名称
ll
使用 sed 命令将 debian/control 中的 gnutls 库改为 openssl 库
sed -i -- 's/libcurl4-gnutls-dev/libcurl4-openssl-dev/' ./debian/control
删除 test 选项
sed -i -- '/TEST\s*=\s*test/d' ./debian/rules
3.2.1.5 编译安装Git
开始编译
dpkg-buildpackage -rfakeroot -b -uc -us
报错 dpkg-checkbuilddeps: error
dpkg-checkbuilddeps: error: Unmet build dependencies: libz-dev gettext libpcre2-dev | libpcre3-dev libexpat1-dev subversion libsvn-perl libyaml-perl tcl libhttp-date-perl | libtime-parsedate-perl libcgi-pm-perl libmailtools-perl cvs cvsps libdbd-sqlite3-perl unzip libio-pty-perl debhelper-compat (= 10) dh-exec (>= 0.7) dh-apache2 asciidoc (>= 8.6.