嵌入式深度学习框架之Tensorflow(四)Tensorflow-Lite运行时环境搭建
参考文档
- http://www.cnblogs.com/jojodru/p/7744630.html 在Ubuntu 16.04上使用bazel交叉编译tensorflow
- https://docs.bazel.build/versions/master/install-ubuntu.html Ubuntu安装Bazel文档
- https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain Building with a custom toolchain
本文档涉及到的目标硬件为全志H8(8核Cortex-A7，Armv7架构)，但是对其他Arm芯片也有一定的借鉴意义，只需要更换交叉编译链即可。
本部分详细介绍了Tensorflow针对Arm-Linux操作系统的源码编译，主要内容是参考《在Ubuntu 16.04上使用bazel交叉编译tensorflow》而来。本文档中介绍的编译方法是使用X86_64机器的Ubuntu环境下进行Tensorflow ARM版本的交叉编译，和网上能查到的大部分(例如参考文档3)直接在树莓派3上编译Tensorflow的方式不同。直接在树莓派3(或其他ARM开发板)上编译会避免很多编译错误，但是编译时间非常长(长达数天)，并且由于编译过程中需要消耗大量内存所以还需要额外用TF卡映射交换空间才能保证编译过程不会因为缺少内存而崩溃。另外直接在树莓派3(或其他ARM开发板)上编译Tensorflow会导致编译出来的文件的适用范围比较小，如果你需要评估多款不同ARM开发板，那么你将花费多余的时间针对每块开发板都做一次编译。所以更好的解决方式是通过bazel进行交叉编译。
开发环境介绍
- 主机操作系统：Ubuntu14.04 64位
- 目标平台：全志H8 CQA83t
- 交叉工具链：gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux
- Tensorflow版本：1.7.0
下载Tensorflow
git clone https://github.com/tensorflow/tensorflow.git或者直接从github上下载稳定版本，并解压源代码。
安装Bazel
参考Bazel官方安装文档的建议，推荐使用apt方式安装Bazel。
安装jdk8
对于Ubuntu16.04：
sudo apt-get install openjdk-8-jdk对于Ubuntu14.04则需要先添加PPA源再进行安装：
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update && sudo apt-get install oracle-java8-installer添加Bazel安装源
sudo apt-get install curl
echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -安装Bazel
sudo apt-get update
sudo apt-get install bazel如果安装过程中出现连接googleapi网站失败的问题，可以尝试多试几次。
升级Bazel
建议安装完Bazel之后进行版本升级
sudo apt-get upgrade bazel设置交叉编译链
# 在/etc/bash.bashrc的最后增加如下指令
# Allwinner H8 CQA83t cross compiler
export ARCH=arm
export PATH=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/:$PATH
export CROSS_COMPILE=arm-linux-gnueabihf-
export CC=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-gcc
export CXX=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-g++
export LD=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-ld
export AR=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-ar
export AS=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-as
export RANLIB=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-ranlib
# 修改完成之后需要重启命令行才能生效
# 你可以通过如下指令来确认交叉编译链是否已经设置好
echo $CC
# 当显示/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-gcc时表示交叉编译链
# 已经设置好
# 当需要更换为本机编译时屏蔽上面的指令即可准备交叉编译链
下面列举一下需要新建和修改的交叉编译相关的Bazel配置脚本：
- WORKSPACE
- arm_compiler
- BUILD
- build_armv8.sh
- build_lite_armv8.sh
- build_armv7.sh
- build_lite_armv7.sh
- cross_toolchain_target_armv8.BUILD
- cross_toolchain_target_armv7.BUILD
- CROSSTOOL
因为全志H8是armv7架构，本文先介绍armv7相关的部分。
修改WORKSPACE
在WORKSPACE添加如下内容：
new_local_repository(
name ='toolchain_target_armv7',
path ='/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux',
build_file = 'arm_compiler/cross_toolchain_target_armv7.BUILD'
)参数说明(按照实际情况修改)：
toolchain_target_armv7：交叉编译器别名
/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux：交叉编译器路径
arm_compiler/cross_toolchain_target_armv7.BUILD：交叉编译器描述文件
新建交叉编译器描述文件
新建cross_toolchain_target_armv7.BUILD：
package(default_visibility = ['//visibility:public'])
filegroup(
name = 'gcc',
srcs = [
'bin/arm-linux-gnueabihf-gcc',
],
)
filegroup(
name = 'ar',
srcs = [
'bin/arm-linux-gnueabihf-ar',
],
)
filegroup(
name = 'ld',
srcs = [
'bin/arm-linux-gnueabihf-ld',
],
)
filegroup(
name = 'nm',
srcs = [
'bin/arm-linux-gnueabihf-nm',
],
)
filegroup(
name = 'objcopy',
srcs = [
'bin/arm-linux-gnueabihf-objcopy',
],
)
filegroup(
name = 'objdump',
srcs = [
'bin/arm-linux-gnueabihf-objdump',
],
)
filegroup(
name = 'strip',
srcs = [
'bin/arm-linux-gnueabihf-strip',
],
)
filegroup(
name = 'as',
srcs = [
'bin/arm-linux-gnueabihf-as',
],
)
filegroup(
name = 'compiler_pieces',
srcs = glob([
'arm-linux-gnueabihf/**',
'libexec/**',
'lib/gcc/arm-linux-gnueabihf/**',
'include/**',
]),
)
filegroup(
name = 'compiler_components',
srcs = [
':gcc',
':ar',
':ld',
':nm',
':objcopy',
':objdump',
':strip',
':as',
],
)交叉编译器描述文件的具体语法参考：https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain
新建CROSSTOOL
CROSSTOOL文件负描述交叉编译器的各种编译选项和链接选项：
major_version: "local"
minor_version: ""
default_target_cpu: "armv7"
default_toolchain {
cpu: "armv7"
toolchain_identifier: "arm-linux-gnueabihf"
}
default_toolchain {
cpu: "k8"
toolchain_identifier: "local"
}
toolchain {
abi_version: "gcc"
abi_libc_version: "glibc_2.21"
builtin_sysroot: ""
compiler: "compiler"
host_system_name: "armv7"
needsPic: true
supports_gold_linker: false
supports_incremental_linker: false
supports_fission: false
supports_interface_shared_objects: false
supports_normalizing_ar: true
supports_start_end_lib: false
supports_thin_archives: true
target_libc: "glibc_2.21"
target_cpu: "armv7"
target_system_name: "armv7"
toolchain_identifier: "arm-linux-gnueabihf"
tool_path { name: "ar" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-ar" }
tool_path { name: "compat-ld" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-ld" }
tool_path { name: "cpp" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-cpp" }
tool_path { name: "dwp" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-dwp" }
tool_path { name: "gcc" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-gcc" }
tool_path { name: "gcov" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-gcov" }
tool_path { name: "ld" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-ld" }
tool_path { name: "nm" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-nm" }
tool_path { name: "objcopy" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-objcopy" }
objcopy_embed_flag: "-I"
objcopy_embed_flag: "binary"
tool_path { name: "objdump" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-objdump" }
tool_path { name: "strip" path: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-strip" }
compiler_flag: "-nostdinc"
compiler_flag: "-isystem"
compiler_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/libc/usr/include"
compiler_flag: "-isystem"
compiler_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/lib/gcc/arm-linux-gnueabihf/4.9.2/include"
compiler_flag: "-isystem"
compiler_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/lib/gcc/arm-linux-gnueabihf/4.9.2/include-fixed"
compiler_flag: "-isystem"
compiler_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/include/c++/4.9.2"
compiler_flag: "-isystem"
compiler_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/include/c++/4.9.2/arm-linux-gnueabihf"
cxx_flag: "-isystem"
cxx_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/libc/usr/include"
cxx_flag: "-isystem"
cxx_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/lib/gcc/arm-linux-gnueabihf/4.9.2/include"
cxx_flag: "-isystem"
cxx_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/lib/gcc/arm-linux-gnueabihf/4.9.2/include-fixed"
cxx_flag: "-isystem"
cxx_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/include/c++/4.9.2"
cxx_flag: "-isystem"
cxx_flag: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/include/c++/4.9.2/arm-linux-gnueabihf"
cxx_flag: "-std=c++11"
cxx_builtin_include_directory: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/libc/usr/include"
cxx_builtin_include_directory: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/lib/gcc/arm-linux-gnueabihf/4.9.2/include"
cxx_builtin_include_directory: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/lib/gcc/arm-linux-gnueabihf/4.9.2/include-fixed"
cxx_builtin_include_directory: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/include/c++/4.9.2"
cxx_builtin_include_directory: "/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/include/c++/4.9.2/arm-linux-gnueabihf"
linker_flag: "-lstdc++"
linker_flag: "-L/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/lib"
linker_flag: "-L/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/lib"
linker_flag: "-L/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/libc/lib"
linker_flag: "-L/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/arm-linux-gnueabihf/libc/usr/lib"
linker_flag: "-Wl,--dynamic-linker=/lib/ld-linux-aarch64.so.1"
# Anticipated future default.
# This makes GCC and Clang do what we want when called through symlinks.
unfiltered_cxx_flag: "-no-canonical-prefixes"
linker_flag: "-no-canonical-prefixes"
# Make C++ compilation deterministic. Use linkstamping instead of these
# compiler symbols.
unfiltered_cxx_flag: "-Wno-builtin-macro-redefined"
unfiltered_cxx_flag: "-D__DATE__=\"redacted\""
unfiltered_cxx_flag: "-D__TIMESTAMP__=\"redacted\""
unfiltered_cxx_flag: "-D__TIME__=\"redacted\""
# Security hardening on by default.
# Conservative choice; -D_FORTIFY_SOURCE=2 may be unsafe in some cases.
# We need to undef it before redefining it as some distributions now have
# it enabled by default.
compiler_flag: "-U_FORTIFY_SOURCE"
compiler_flag: "-fstack-protector"
compiler_flag: "-fPIE"
linker_flag: "-pie"
linker_flag: "-Wl,-z,relro,-z,now"
# Enable coloring even if there's no attached terminal. Bazel removes the
# escape sequences if --nocolor is specified.
compiler_flag: "-fdiagnostics-color=always"
# All warnings are enabled. Maybe enable -Werror as well?
compiler_flag: "-Wall"
# Enable a few more warnings that aren't part of -Wall.
compiler_flag: "-Wunused-but-set-parameter"
# But disable some that are problematic.
compiler_flag: "-Wno-free-nonheap-object" # has false positives
# Keep stack frames for debugging, even in opt mode.
compiler_flag: "-fno-omit-frame-pointer"
# Stamp the binary with a unique identifier.
linker_flag: "-Wl,--build-id=md5"
linker_flag: "-Wl,--hash-style=gnu"
compilation_mode_flags {
mode: DBG
# Enable debug symbols.
compiler_flag: "-g"
}
compilation_mode_flags {
mode: OPT
# No debug symbols.
# Maybe we should enable https://gcc.gnu.org/wiki/DebugFission for opt or
# even generally? However, that can't happen here, as it requires special
# handling in Bazel.
compiler_flag: "-g0"
# Conservative choice for -O
# -O3 can increase binary size and even slow down the resulting binaries.
# Profile first and / or use FDO if you need better performance than this.
compiler_flag: "-O3"
# Disable assertions
compiler_flag: "-DNDEBUG -mcpu=cortex-a7 -mfpu=neon"
# Removal of unused code and data at link time
# Uncomment fllowing flag when software deploy finally
# compiler_flag: "-ffunction-sections"
# compiler_flag: "-fdata-sections"
# linker_flag: "-Wl,--gc-sections"
}
}
toolchain {
toolchain_identifier: "local"
abi_libc_version: "local"
abi_version: "local"
builtin_sysroot: ""
compiler: "compiler"
compiler_flag: "-U_FORTIFY_SOURCE"
compiler_flag: "-D_FORTIFY_SOURCE=2"
compiler_flag: "-fstack-protector"
compiler_flag: "-Wall"
compiler_flag: "-Wl,-z,-relro,-z,now"
compiler_flag: "-B/usr/bin"
compiler_flag: "-B/usr/bin"
compiler_flag: "-Wunused-but-set-parameter"
compiler_flag: "-Wno-free-nonheap-object"
compiler_flag: "-fno-omit-frame-pointer"
compiler_flag: "-isystem"
compiler_flag: "/usr/include"
cxx_builtin_include_directory: "/usr/include/c++/4.8.5"
cxx_builtin_include_directory: "/usr/include/c++/4.8"
cxx_builtin_include_directory: "/usr/lib/gcc/x86_64-linux-gnu/4.8/include"
cxx_builtin_include_directory: "/usr/include/x86_64-linux-gnu/c++/4.8"
cxx_builtin_include_directory: "/usr/include/c++/4.8/backward"
cxx_builtin_include_directory: "/usr/lib/gcc/x86_64-linux-gnu/4.8/include"
cxx_builtin_include_directory: "/usr/local/include"
cxx_builtin_include_directory: "/usr/lib/gcc/x86_64-linux-gnu/4.8.5/include-fixed"
cxx_builtin_include_directory: "/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed"
cxx_builtin_include_directory: "/usr/include/x86_64-linux-gnu"
cxx_builtin_include_directory: "/usr/include"
cxx_flag: "-std=c++11"
host_system_name: "local"
linker_flag: "-lstdc++"
linker_flag: "-lm"
linker_flag: "-Wl,-no-as-needed"
linker_flag: "-B/usr/bin"
linker_flag: "-B/usr/bin"
linker_flag: "-pass-exit-codes"
needsPic: true
objcopy_embed_flag: "-I"
objcopy_embed_flag: "binary"
supports_fission: false
supports_gold_linker: false
supports_incremental_linker: false
supports_interface_shared_objects: false
supports_normalizing_ar: false
supports_start_end_lib: false
supports_thin_archives: false
target_cpu: "k8"
target_libc: "local"
target_system_name: "local"
unfiltered_cxx_flag: "-fno-canonical-system-headers"
unfiltered_cxx_flag: "-Wno-builtin-macro-redefined"
unfiltered_cxx_flag: "-D__DATE__=\"redacted\""
unfiltered_cxx_flag: "-D__TIMESTAMP__=\"redacted\""
unfiltered_cxx_flag: "-D__TIME__=\"redacted\""
tool_path {name: "ar" path: "/usr/bin/ar" }
tool_path {name: "cpp" path: "/usr/bin/cpp" }
tool_path {name: "dwp" path: "/usr/bin/dwp" }
tool_path {name: "gcc" path: "/usr/bin/gcc" }
tool_path {name: "gcov" path: "/usr/bin/gcov" }
tool_path {name: "ld" path: "/usr/bin/ld" }
tool_path {name: "nm" path: "/usr/bin/nm" }
tool_path {name: "objcopy" path: "/usr/bin/objcopy" }
tool_path {name: "objdump" path: "/usr/bin/objdump" }
tool_path {name: "strip" path: "/usr/bin/strip" }
compilation_mode_flags {
mode: DBG
compiler_flag: "-g"
}
compilation_mode_flags {
mode: OPT
compiler_flag: "-g0"
compiler_flag: "-O3"
compiler_flag: "-DNDEBUG"
compiler_flag: "-ffunction-sections"
compiler_flag: "-fdata-sections"
linker_flag: "-Wl,--gc-sections"
}
linking_mode_flags { mode: DYNAMIC }
}新建BUILD
package(default_visibility = ["//visibility:public"])
cc_toolchain_suite(
name = "toolchain",
toolchains = {
"armv7|compiler": ":cc-compiler-armv7",
"k8|compiler": ":cc-compiler-local",
},
)
filegroup(
name = "empty",
srcs = [],
)
filegroup(
name = "arm_linux_all_files",
srcs = [
"@toolchain_target_armv7//:compiler_pieces",
],
)
cc_toolchain(
name = "cc-compiler-local",
all_files = ":empty",
compiler_files = ":empty",
cpu = "local",
dwp_files = ":empty",
dynamic_runtime_libs = [":empty"],
linker_files = ":empty",
objcopy_files = ":empty",
static_runtime_libs = [":empty"],
strip_files = ":empty",
supports_param_files = 1,
)
cc_toolchain(
name = "cc-compiler-armv7",
all_files = ":arm_linux_all_files",
compiler_files = ":arm_linux_all_files",
cpu = "armv7",
dwp_files = ":empty",
dynamic_runtime_libs = [":empty"],
linker_files = ":arm_linux_all_files",
objcopy_files = "arm_linux_all_files",
static_runtime_libs = [":empty"],
strip_files = "arm_linux_all_files",
supports_param_files = 1,
)添加nsync的交叉编译支持
因为nsync官方的交叉编译器可能和你自带的交叉编译器不是一致的，所以需要你改动nsync的编译设置。
问题：如何找到nsync的配置代码位置？
答案：~/.cache/bazel/_bazel_root/79bf1a12876b6ceef4e98e2416c50946/external/nsync
79bf1a12876b6ceef4e98e2416c50946是一个自动生成的随机文件夹，如果有多个文件夹的话可以通过文件夹的创建时间来判断。
修改/nsync/BUILD,因为nsync添加一个新的编译器需要添加多个位置的代码，所以分段显示需要修改的地方：
// 添加armv7 & armv8 config_setting
config_setting(
name = "armv7",
values = {"cpu": "armeabi-v7a"},
)
config_setting(
name = "armv8",
values = {"cpu": "arm64-v8a"},
)
//
# Compilation options that apply to both C++11 and C.
NSYNC_OPTS_GENERIC = [ # select({
# Select the CPU architecture include directory.
# This select() has no real effect in the C++11 build, but satisfies a
# #include that would otherwise need a #if.
# ":gcc_linux_x86_32_1": ["-I" + pkg_path_name() + "/platform/x86_32"],
# ":gcc_linux_x86_64_1": ["-I" + pkg_path_name() + "/platform/x86_64"],
# ":gcc_linux_x86_64_2": ["-I" + pkg_path_name() + "/platform/x86_64"],
# ":gcc_linux_aarch64": ["-I" + pkg_path_name() + "/platform/aarch64"],
# ":gcc_linux_ppc64": ["-I" + pkg_path_name() + "/platform/ppc64"],
# ":gcc_linux_s390x": ["-I" + pkg_path_name() + "/platform/s390x"],
# ":clang_macos_x86_64": ["-I" + pkg_path_name() + "/platform/x86_64"],
# ":freebsd": ["-I" + pkg_path_name() + "/platform/x86_64"],
# ":ios_x86_64": ["-I" + pkg_path_name() + "/platform/x86_64"],
# ":armv8": ["-I" + pkg_path_name() + "/platform/aarch64"],
# ":android_x86_32": ["-I" + pkg_path_name() + "/platform/x86_32"],
# ":android_x86_64": ["-I" + pkg_path_name() + "/platform/x86_64"],
# ":android_armeabi": ["-I" + pkg_path_name() + "/platform/arm"],
# ":android_arm": ["-I" + pkg_path_name() + "/platform/arm"],
# ":android_arm64": ["-I" + pkg_path_name() + "/platform/aarch64"],
# ":msvc_windows_x86_64": ["-I" + pkg_path_name() + "/platform/x86_64"],
# }) + [
"-I" + pkg_path_name() + "/public",
"-I" + pkg_path_name() + "/internal",
"-I" + pkg_path_name() + "/platform/posix",
] + select({
":msvc_windows_x86_64": [
],
":freebsd": ["-pthread"],
"//conditions:default": [
"-D_POSIX_C_SOURCE=200809L",
"-pthread",
],
})
// 添加armv7 & armv8 NSYNC_OPTS
# Options for C build, rather then C++11 build.
NSYNC_OPTS = select({
# Select the OS include directory.
":gcc_linux_x86_32_1": ["-I" + pkg_path_name() + "/platform/linux"],
":gcc_linux_x86_64_1": ["-I" + pkg_path_name() + "/platform/linux"],
":gcc_linux_x86_64_2": ["-I" + pkg_path_name() + "/platform/linux"],
":gcc_linux_aarch64": ["-I" + pkg_path_name() + "/platform/linux"],
":gcc_linux_ppc64": ["-I" + pkg_path_name() + "/platform/linux"],
":gcc_linux_s390x": ["-I" + pkg_path_name() + "/platform/linux"],
":clang_macos_x86_64": ["-I" + pkg_path_name() + "/platform/macos"],
":freebsd": ["-I" + pkg_path_name() + "/platform/freebsd"],
":ios_x86_64": ["-I" + pkg_path_name() + "/platform/macos"],
":android_x86_32": ["-I" + pkg_path_name() + "/platform/linux"],
":android_x86_64": ["-I" + pkg_path_name() + "/platform/linux"],
":android_armeabi": ["-I" + pkg_path_name() + "/platform/linux"],
":android_arm": ["-I" + pkg_path_name() + "/platform/linux"],
":android_arm64": ["-I" + pkg_path_name() + "/platform/linux"],
":armv7": ["-I" + pkg_path_name() + "/platform/linux"],
":armv8": ["-I" + pkg_path_name() + "/platform/linux"],
":msvc_windows_x86_64": ["-I" + pkg_path_name() + "/platform/win32"],
"//conditions:default": [],
}) + select({
# Select the compiler include directory.
":gcc_linux_x86_32_1": ["-I" + pkg_path_name() + "/platform/gcc"],
":gcc_linux_x86_64_1": ["-I" + pkg_path_name() + "/platform/gcc"],
":gcc_linux_x86_64_2": ["-I" + pkg_path_name() + "/platform/gcc"],
":gcc_linux_aarch64": ["-I" + pkg_path_name() + "/platform/gcc"],
":gcc_linux_ppc64": ["-I" + pkg_path_name() + "/platform/gcc"],
":gcc_linux_s390x": ["-I" + pkg_path_name() + "/platform/gcc"],
":clang_macos_x86_64": ["-I" + pkg_path_name() + "/platform/clang"],
":freebsd": ["-I" + pkg_path_name() + "/platform/clang"],
":ios_x86_64": ["-I" + pkg_path_name() + "/platform/clang"],
":android_x86_32": ["-I" + pkg_path_name() + "/platform/gcc"],
":android_x86_64": ["-I" + pkg_path_name() + "/platform/gcc"],
":android_armeabi": ["-I" + pkg_path_name() + "/platform/gcc"],
":android_arm": ["-I" + pkg_path_name() + "/platform/gcc"],
":android_arm64": ["-I" + pkg_path_name() + "/platform/gcc"],
":armv7": ["-I" + pkg_path_name() + "/platform/gcc"],
":armv8": ["-I" + pkg_path_name() + "/platform/gcc"],
":msvc_windows_x86_64": ["-I" + pkg_path_name() + "/platform/msvc"],
}) + select({
# Apple deprecated their atomics library, yet recent versions have no
# working version of stdatomic.h; so some recent versions need one, and
# other versions prefer the other. For the moment, just ignore the
# depreaction.
":clang_macos_x86_64": ["-Wno-deprecated-declarations"],
"//conditions:default": [],
}) + NSYNC_OPTS_GENERIC
// 添加armv7 & armv8 NSYNC_SRC_PLATFORM
NSYNC_SRC_PLATFORM = select({
":gcc_linux_x86_32_1": NSYNC_SRC_LINUX,
":gcc_linux_x86_64_1": NSYNC_SRC_LINUX,
":gcc_linux_x86_64_2": NSYNC_SRC_LINUX,
":gcc_linux_aarch64": NSYNC_SRC_LINUX,
":gcc_linux_ppc64": NSYNC_SRC_LINUX,
":gcc_linux_s390x": NSYNC_SRC_LINUX,
":clang_macos_x86_64": NSYNC_SRC_MACOS,
":freebsd": NSYNC_SRC_FREEBSD,
":ios_x86_64": NSYNC_SRC_MACOS,
":android_x86_32": NSYNC_SRC_ANDROID,
":android_x86_64": NSYNC_SRC_ANDROID,
":android_armeabi": NSYNC_SRC_ANDROID,
":android_arm": NSYNC_SRC_ANDROID,
":android_arm64": NSYNC_SRC_ANDROID,
":armv7": NSYNC_SRC_LINUX,
":armv8": NSYNC_SRC_LINUX,
":msvc_windows_x86_64": NSYNC_SRC_WINDOWS,
})
// 添加armv7 & armv8 NSYNC_TEST_SRC_PLATFORM
NSYNC_TEST_SRC_PLATFORM = select({
":gcc_linux_x86_32_1": NSYNC_TEST_SRC_LINUX,
":gcc_linux_x86_64_1": NSYNC_TEST_SRC_LINUX,
":gcc_linux_x86_64_2": NSYNC_TEST_SRC_LINUX,
":gcc_linux_aarch64": NSYNC_TEST_SRC_LINUX,
":gcc_linux_ppc64": NSYNC_TEST_SRC_LINUX,
":gcc_linux_s390x": NSYNC_TEST_SRC_LINUX,
":clang_macos_x86_64": NSYNC_TEST_SRC_MACOS,
":freebsd": NSYNC_TEST_SRC_FREEBSD,
":ios_x86_64": NSYNC_TEST_SRC_MACOS,
":android_x86_32": NSYNC_TEST_SRC_ANDROID,
":android_x86_64": NSYNC_TEST_SRC_ANDROID,
":android_armeabi": NSYNC_TEST_SRC_ANDROID,
":android_arm": NSYNC_TEST_SRC_ANDROID,
":android_arm64": NSYNC_TEST_SRC_ANDROID,
":armv7": NSYNC_TEST_SRC_LINUX,
":armv8": NSYNC_TEST_SRC_LINUX,
":msvc_windows_x86_64": NSYNC_TEST_SRC_WINDOWS,
})运行configure配置脚本
cd tensorflow-1.7.0
./configure配置过程中注意以下选项的设置：
python及其库的位置：再安装mxnet的一些python库之后默认的python库位置会出现错误，需要重新设置为/usr/local/lib/python2.7开头的路径
optimization flags：默认为-march=native，对于交叉编译而言需要进行修改，可以根据实际芯片架构进行设置，如-march=armv7-a
其他的选项都选择no即可
交叉编译Tensorflow主体部分
bazel build --copt="-fPIC" --copt="-march=armv7-a" --cxxopt="-fPIC" --cxxopt="-march=armv7-a" --verbose_failures --crosstool_top=//arm_compiler:toolchain --cpu=armv7 --config=opt tensorflow/examples/label_image/...bazel命令可以通过–jobs选项来指定编译线程数，如果不指定则采用CPU核心数x2的线程数
编译过程中会可能会出现多种错误，下面将一一列举并解决：
(1) pcre编译错误
(2) boringssl编译错误
(3) arm版本python头文件缺失
(4) numpy头文件缺失
(5) tensorflow核心编译错误
在这里值得注意的是，上述错误跟tensorflow的版本有相当大的关系，这里只列举1.7.0版本编译会出现的问题。
完成编译之后将会生成如下目标文件：
- bazel-bin/tensorflow/libtensorflow_framework.so
- bazel-bin/tensorflow/examples/label_image
交叉编译Tensorflow-Lite
交叉编译过程参考tensorfow/contrib/lite/g3doc/rpi.md
在编译之前先对涉及到多线程的相关代码进行修改，Tensorflow-Lite默认的线程池数量为4：
const Eigen::ThreadPoolDevice& GetThreadPoolDevice() {
const int thread_count = 4;
static Eigen::ThreadPool* tp = new Eigen::ThreadPool(thread_count);
static EigenThreadPoolWrapper* thread_pool_wrapper =
new EigenThreadPoolWrapper(tp);
static Eigen::ThreadPoolDevice* device =
new Eigen::ThreadPoolDevice(thread_pool_wrapper, thread_count);
return *device;
}将上述代码修改为：
const Eigen::ThreadPoolDevice& GetThreadPoolDevice() {
int thread_count = 1;
const char *val = getenv("OMP_NUM_THREADS");
if (val != nullptr) {
thread_count = atoi(val);
}
static Eigen::ThreadPool* tp = new Eigen::ThreadPool(thread_count);
static EigenThreadPoolWrapper* thread_pool_wrapper =
new EigenThreadPoolWrapper(tp);
static Eigen::ThreadPoolDevice* device =
new Eigen::ThreadPoolDevice(thread_pool_wrapper, thread_count);
return *device;
}./tensorflow/contrib/lite/download_dependencies.sh
./tensorflow/contrib/lite/build_rpi_lib.shdownload_dependencies.sh只需要执行一次即可，不需要多次执行。
完成编译之后将会生成如下目标文件：
- tensorflow/contrib/lite/gen/lib/rpi_armv7/libtensorflow-lite.a
- tensorflow/contrib/lite/gen/bin/rpi_armv7/benchmark_model
交叉编译Tensorflow-Lite的label_image测试工具
bazel build --copt="-fPIC" --copt="-march=armv7-a" --cxxopt="-fPIC" --cxxopt="-march=armv7-a" --verbose_failures --crosstool_top=//arm_compiler:toolchain --cpu=armv7 --config=opt //tensorflow/contrib/lite/examples/label_image:label_image本地编译Toco模型转换工具
Tensorflow-Lite为了最小化运行时环境，使用了Flatbuffer这种轻量级的数据存储格式。所以在使用Tensorflow-Lite进行模型测试前需要使用TOCO工具进行模型转换。
注意：TOCO转换工具和模型预测无关，建议运行在X86机器上
为了不和交叉编译的目录相互影响，拷贝一份tensorflow到另外的目录，并重新进行编译前的配置，并注意设置-march=native选项，其他选项和交叉编译时保持一致。
屏蔽/etc/bash.bashrc中设置的交叉编译链选项，并且重启命令行
bazel build tensorflow/contrib/lite/toco:toco完成编译之后将会生成如下目标文件：
- bazel-bin/tensorflow/contrib/lite/toco/toco
注意：toco工具编译出来之后尽量不要移动位置，推荐原地执行。
至此Tensorflow-Lite的交叉编译过程已经完成，之后将会有一篇ARM端模型部署&测试的文章。