linux-aarch64编译安装tensorflow1.14.0
linux-aarch64 编译安装tensorflow
- 1. 明确版本
- 2. 管理版本
- 3. 编译安装Bazel-0.24.1
-
- 3.1 pip安装python相关包
- 3.2 编译Bazel
- 4. 编译tensorflow1.14.0
-
- 4.1 源码下载
- 4.2 设置编译参数
- 4.3 编译tensorflow
-
- 4.3.1 依赖下载失败
- 4.3.2 C++ compilation of rule '@grpc//:gpr_base' failed (Exit 1):
- 4.3.3 depthwiseconv_uint8_3x3_filter.h:3957:58: error:
- 5. 构建.whl文件，安装tensorflow
- 6. 安装tensorflow1.13.1的坑
-
- 6.1 编译时的坑
- 6.2 安装时的坑
1. 明确版本
首先要明确你的系统，本文是在ubuntu20.04（内核架构为aarch64）上安装tensorflow1.14.0，由于在网上找到的.whl文件不好使，选择编译安装。参考tensorflow官网，以下是对应版本。
| Tensorflow | GCC | Bazel | Python | Numpy |
|---|---|---|---|---|
1.14.0 |
5.3.1 |
0.24.1 |
3.7.6 |
1.16.5 |
1.13.1 |
5.3.1 |
0.19.2 |
3.5.3 |
1.16.5 |
Note：tensorflow官网上给的gcc版本为4.8，在编译的时候会由于C89和C99的区别导致编译问题，故在此选用了5.3.1。
Note：numpy的版本最好不要高于1.19.0。
2. 管理版本
由于编译过程中各种版本选择不是一次就能成功的，本文采用ubuntu的update-alternatives命令对Python以及GCC版本进行管理。参考链接：https://blog.csdn.net/a1809032425/article/details/122729307。
There are 4 choices for the alternative python (providing /usr/bin/python).
Selection Path Priority Status
------------------------------------------------------------
0 /usr/local/bin/python3.5 3 auto mode
1 /usr/bin/python2.7 1 manual mode
2 /usr/bin/python3.8 1 manual mode
3 /usr/local/bin/python3.5 3 manual mode
* 4 /usr/local/bin/python3.7 1 manual mode
Press <enter> to keep the current choice[*], or type selection number:
There are 5 choices for the alternative gcc (providing /usr/bin/gcc).
Selection Path Priority Status
------------------------------------------------------------
0 /usr/bin/aarch64-linux-gnu-gcc-4.8 2 auto mode
1 /usr/bin/aarch64-linux-gnu-gcc-4.8 2 manual mode
* 2 /usr/bin/aarch64-linux-gnu-gcc-5 1 manual mode
3 /usr/bin/aarch64-linux-gnu-gcc-9 1 manual mode
4 /usr/bin/gcc-4.8 1 manual mode
5 /usr/bin/gcc-9 1 manual mode
Press <enter> to keep the current choice[*], or type selection number:
3. 编译安装Bazel-0.24.1
3.1 pip安装python相关包
若系统中有多个版本的python，一定要管理好，用对应版本的pip去安装相应的包。这里没有按照tensorflow官网中那样使用--user安装python包，目的是为了让不同版本的python将包装在自己的文件夹下从而不造成冲突。
sudo pip3.7 install numpy==1.16.5 wheel -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
sudo pip3.7 install keras_preprocessing --no-deps -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
3.2 编译Bazel
同样地，通过脚本安装Bazel不好使，所以仍然采取编译安装bazel的方式。具体参考Bazel官方文档中的Build Bazel from scratch (bootstrapping)。
Note：从Github上下载Bazel源码的时候一定要注意下载形如bazel-的文件，否则可能会出现无法编译的问题。-dist.zip
在编译Bazel的时候，出现了error: ambiguating new declaration of ‘long int gettid()’,这是在编译grpc的时候产生的问题，为此需要修改如下两个文件（将gettid更改为sys_gettid）。
vim bazel/third_party/grpc/src/core/lib/gpr/log_linux.cc:
/*修改之前*/
43:static long gettid(void) { return syscall(__NR_gettid); }
73: if (tid == 0) tid = gettid();
/*修改之后*/
43:static long sys_gettid(void) { return syscall(__NR_gettid); }
73: if (tid == 0) tid = sys_gettid();
vim bazel/third_party/grpc/src/core/lib/gpr/log_posix.cc:
/*修改之前*/
33：static intptr_t gettid(void) { return (intptr_t)pthread_self(); }
86： gpr_asprintf(&prefix, "%s%s.%09d %7tu %s:%d]",
gpr_log_severity_string(args->severity), time_buffer,
(int)(now.tv_nsec), gettid(), display_file, args->line);
/*修改之后*/
33：static intptr_t sys_gettid(void) { return (intptr_t)pthread_self(); }
86： gpr_asprintf(&prefix, "%s%s.%09d %7tu %s:%d]",
gpr_log_severity_string(args->severity), time_buffer,
(int)(now.tv_nsec), sys_gettid(), display_file, args->line);
修改完之后，再次输入env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh命令编译，即可成功安装。
Note：Bazel的编译需要配置java环境，jdk最好选用1.8。
4. 编译tensorflow1.14.0
4.1 源码下载
终于要编译tensorflow了，首先要从github上下载1.14.0的源码，下载地址https://github.com/tensorflow/tensorflow/releases?q=1.14.0&expanded=true。
4.2 设置编译参数
解压后进入tensorflow1.14.0文件夹，然后运行./configure脚本设置编译参数
Extracting Bazel installation...
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.24.1- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python]:
Found possible Python library paths:
/usr/local/lib/python3.7/site-packages
Please input the desired Python library path to use. Default is [/usr/local/lib/python3.7/site-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]:
XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]:
No CUDA support will be enabled for TensorFlow.
Do you wish to download a fresh release of clang? (Experimental) [y/N]:
Clang will not be downloaded.
Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=gdr # Build with GDR support.
--config=verbs # Build with libverbs support.
--config=ngraph # Build with Intel nGraph support.
--config=numa # Build with NUMA support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=noignite # Disable Apache Ignite support.
--config=nokafka # Disable Apache Kafka support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished
Note：建议都选否，cuda如果有可以选。
4.3 编译tensorflow
输入以下命令进行编译，--local_resources是指定系统给编译过程中分配的资源，分别分配内存大小（MB）、CPU核心数（个）、可利用IO的工作站（平均为1.0）。
bazel build --conlyopt="-std=gnu99" --conlyopt="-w" --local_resources 14436,4.0,1.0 //tensorflow/tools/pip_package:build_pip_package
4.3.1 依赖下载失败
编译tensorflow的时候需要许多的依赖包，由于本文不是离线编译，依赖包可以通过联网下载，但有一些包会下载失败，为此，需要手动下载这些包。并将./WORKSPACE、./third_party/icu/workspace.bzl、./tensorflow/workspace.bzl、
./third_party/flatbuffers/workspace.bzl4个文件中的相应包的路径换成下载包的路径。
Note：下载不下来包的大多是网络的问题，当然可以不去手动下载，重新尝试几次。
4.3.2 C++ compilation of rule ‘@grpc//:gpr_base’ failed (Exit 1):
这个错误与Bazel编译失败类似，也是在编译grpc中产生的错误，类似地，修改/.cache/bazel/_bazel_wsn/a7132f72a8b641c1ebbdd6a7fd1fd5fb/external/grpc/src/core/lib/gpr/log_linux.cc与home/wsn/.cache/bazel/_bazel_wsn/a7132f72a8b641c1ebbdd6a7fd1fd5fb/external/grpc/src/core/lib/gpr/log_posix.cc两个文件中的gettid为’sys_gettid’即可成功编译。
4.3.3 depthwiseconv_uint8_3x3_filter.h:3957:58: error:
在编译该文件的时候，产生 cannot convert ‘uint8x16_t {aka __vector(16) unsigned char}’ to ‘const int8x16_t {aka const __vector(16) signed char}’的错误，通过修改./tensorflow/lite/build_def.bzl文件添加如下内容即可成功编译。参考链接https://github.com/tensorflow/tensorflow/pull/29515。
"/DTF_COMPILE_LIBRARY",
"/wd4018", # -Wno-sign-compare
],
+ str(Label("//tensorflow:linux_aarch64")): [
+ "-flax-vector-conversions",
+ "-fomit-frame-pointer",
+ ],
"//conditions:default": [
"-Wno-sign-compare",
],
之后就可以等待tensorflow编译完成了，这是一个漫长的过程…
5. 构建.whl文件，安装tensorflow
输入下面命令构建.whl文件，输出在/tmp/tensorflow_pkg文件夹下
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
进入输出目录，输入下面进行安装tensorflow1.14.0
sudo pip3.7 install tensorflow-1.14.0-cp37-cp37m-linux_aarch64.whl -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
测试import tensorflow,出现错误`TypeError: Descriptors cannot not be created directly，按照下面提示降低protobuf的版本。
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
sudo pip3.7 install tensorflow-1.14.0-cp37-cp37m-linux_aarch64.whl -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
重新测试，安装成功。
>>> import tensorflow
>>> print(tensorflow.__version__)
1.14.0
6. 安装tensorflow1.13.1的坑
6.1 编译时的坑
除了上述4.3.1和4.3.2的问题外，由于tensorflow1.13.1中aws-sdk对linux-aarch64支持不好， 会在编译时产生ImportError: ....undefined symbol: _ZN3Aws11Environment6GetEnvB5cxx11EPKc错误。解决方法是通过修改./tensorflow/BUILD文件添加如下内容：
visibility = ["//visibility:public"],
)
+ config_setting(
+ name = "linux_aarch64",
+ values = {"cpu": "aarch64"},
+ visibility = ["//visibility:public"],
+ )
config_setting(
name = "linux_x86_64",
values = {"cpu": "k8"},
修改./third_party/aws/BUILD.bazel,添加如下内容：
cc_library(
name = "aws",
srcs = select({
+ "@org_tensorflow//tensorflow:linux_aarch64": glob([
+ "aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",
+ ]),
"@org_tensorflow//tensorflow:linux_x86_64": glob([
"aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",
]),
参考链接:https://github.com/tensorflow/tensorflow/pull/22856。
6.2 安装时的坑
在安装tensorflow的时候，依赖库h5py安装失败,本想找.whl文件安装，找了半天发现aarch64架构只能找到python3.7版本，于是放弃了tensorflow1.13.1，重新编译python3.7，bazel-0.24.1，tensorflow1.14.0，完成安装。但好像也有其他解决方法？当时一气之下重新安装了，没有动脑子…
Note：根据tensorflow官方文档，用python3.7编译tensorflow1.13.1也可以，但没有尝试。