Tensorflow2.3r CPU编译过程记录

编译Tensorflow2.2.0 CPU 版本

1. 安装 Anaconda3-2020.02-Windows-x86_64.exe （管理python）

2. 安装 Visual Studio 2019 配置VS2019编译环境 (Windows 需要) 添加系统变量 BAZEL_VC C:\Program Files (x86)\ Microsoft Visual Studio\2019\Community\VC
   BAZEL_VS C:\Program Files (x86)\ Microsoft Visual Studio\2019\Community

3. 使用conda创建python虚拟环境 conda create -n xxx python=3.7

4. 转到新创建的虚拟环境下 conda activate xxx

5. 安装编译过程需要用到的模块(如果下载速度慢,可以修改pip源) pip install numpy six keras_preprocessing
   conda install protobuf

6. 安装 git 和 bazel2.0.0
   windows: 直接配置bazel.exe的环境变量
   linux : sudo rm -rf ~/.bazel
   sudo rm -rf ~/bin
   sudo rm -rf /usr/bin/bazel
   sudo cp bazel /usr/bin/bazel
   sudo chmod a+x /usr/bin/bazel

7. 安装 msys2 pacman -S zip unzip patch diffutils git (Windows需要)

8. 下载tensorflow源码 https://github.com/tensorflow/tensorflow/releases/tag/v2.3r

9. cd tensorflow

10. 在创建的python虚拟环境下 python configure.py

                                                Please specify the location of python. [Default is xxx\Anaconda3\envs\pythonVersion3\python.exe]:
    
    
                                                Found possible Python library paths:
                                                xxx\Anaconda3\envs\pythonVersion3\lib\site-packages
                                                Please input the desired Python library path to use.  Default is [xxx\Anaconda3\envs\pythonVersion3\lib\site-packages]
    
                                                Do you wish to build TensorFlow with ROCm support? [y/N]:
                                                No ROCm support will be enabled for TensorFlow.
    
                                                Do you wish to build TensorFlow with CUDA support? [y/N]:
                                                No CUDA support will be enabled for TensorFlow.
    
                                                Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is /arch:AVX]:
    
    
                                                Would you like to override eigen strong inline for some C++ compilation to reduce the compilation time? [Y/n]:
                                                Not overriding eigen strong inline, some compilations could take more than 20 mins.
    
                                                Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
                                                        --config=mkl            # Build with MKL support.
                                                        --config=monolithic     # Config for mostly static monolithic build.
                                                        --config=ngraph         # Build with Intel nGraph support.
                                                        --config=numa           # Build with NUMA support.
                                                        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
                                                        --config=v2             # Build TensorFlow 2.x instead of 1.x.
                                                Preconfigured Bazel build configs to DISABLE default on features:
                                                        --config=noaws          # Disable AWS S3 filesystem support.
                                                        --config=nogcp          # Disable GCP support.
                                                        --config=nohdfs         # Disable HDFS support.
                                                        --config=nonccl         # Disable NVIDIA NCCL support.
    

11. 开始使用bazel编译
    编译tensorflow_cc.dll : bazel --output_user_root="…/output_user_root" build --noincompatible_do_not_split_linking_cmdline --verbose_failures --config=opt //tensorflow:tensorflow_cc.dll
    编译tensorflowlite.dll : bazel --output_user_root="…/output_user_root" build --noincompatible_do_not_split_linking_cmdline --verbose_failures --config=opt //tensorflow/lite:tensorflowlite.dll

注: 编译会占用大量的CPU,建议不要在个人的电脑编译
编译过程会下载第三方库,放到output_user_root 中***此部分的库不要删掉***,使用tensorflow库时需要添加部分第三库的.lib

使用tensorflow lite模块需要在VS中配置以下内容
添加的头文件:
tensorflow2.3
flatbuffers

添加的库路径:
tensorflow2.3\tensorflow\lite
external

需要添加的链接库:
tensorflowlite.dll.if.lib
allocation.lib
util.lib
external_cpu_backend_context.lib
arena_planner.lib
simple_memory_arena.lib
c/common.lo.lib
core/api/api.lib
delegates\nnapi\nnapi_delegate.lib
experimental\resource\resource.lib
nnapi\nnapi_util.lib
nnapi\nnapi_implementation.lib
framework_lib.lo.lib
kernels\builtin_ops_all_linked.lo.lib
minimal_logging.lib
string_util.lib
kernels\builtin_op_kernels.lib
kernels\internal\audio_utils.lib
kernels\internal\kernel_utils.lib
kernels\internal\quantization_util.lib
kernels\internal\tensor_utils.lib
kernels\internal\portable_tensor_utils.lib
kernels\internal\transpose_utils.lib
kernels\internal\sse_tensor_utils.lib
kernels\kernel_util.lib
kernels\lstm_eval.lib
kernels\cpu_backend_context.lib
kernels\cpu_backend_gemm.lib
kernels\eigen_support.lib
tools\optimize\sparsity\format_converter.lib
fft2d\fft2d.lib
farmhash_archive\farmhash.lib
flatbuffers\src\flatbuffers.lib

编译android tflite:

python configure.py
You have bazel 3.2.0 installed.
Please specify the location of python. [Default is D:\SoftWare\Anaconda3\envs\pythonVersion3\python.exe]:

Found possible Python library paths:
D:\SoftWare\Anaconda3\envs\pythonVersion3\lib\site-packages
Please input the desired Python library path to use. Default is [D:\SoftWare\Anaconda3\envs\pythonVersion3\lib\site-packages]

Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]:
No CUDA support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option “–config=opt” is specified [Default is /arch:AVX]:

Would you like to override eigen strong inline for some C++ compilation to reduce the compilation time? [Y/n]:
Eigen strong inline overridden.

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: y
Searching for NDK and SDK installations.

Please specify the home path of the Android NDK to use. [Default is C:/Users/Admin/AppData/Roaming/Android/Sdk/ndk-bundle]: D:/SoftWare/Android/Sdk/ndk/21.3.6528147

WARNING: The NDK version in D:/SoftWare/Android/Sdk/ndk/21.3.6528147 is 21, which is not supported by Bazel (officially supported versions: [10, 11, 12, 13, 14, 15, 16, 17, 18]). Please use another version. Compiling Android targets may result in confusing errors.

Please specify the (min) Android NDK API level to use. [Available levels: [‘16’, ‘17’, ‘18’, ‘19’, ‘21’, ‘22’, ‘23’, ‘24’, ‘26’, ‘27’, ‘28’, ‘29’, ‘30’]] [Default is 21]: 16

Please specify the home path of the Android SDK to use. [Default is C:/Users/Admin/AppData/Roaming/Android/Sdk]: D:/SoftWare/Android/Sdk

Please specify the Android SDK API level to use. [Available levels: [‘30’, ‘30.backup’]] [Default is 30.backup]: 30

Please specify an Android build tools version to use. [Available versions: [‘30.0.0’]] [Default is 30.0.0]:

Preconfigured Bazel build configs. You can use any of the below by adding “–config=<>” to your build command. See .bazelrc for more details.
–config=mkl # Build with MKL support.
–config=monolithic # Config for mostly static monolithic build.
–config=ngraph # Build with Intel nGraph support.
–config=numa # Build with NUMA support.
–config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
–config=v2 # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
–config=noaws # Disable AWS S3 filesystem support.
–config=nogcp # Disable GCP support.
–config=nohdfs # Disable HDFS support.
–config=nonccl # Disable NVIDIA NCCL support.

bazel build -c opt --cxxopt=–std=c++11 --config=android_arm64 //tensorflow/lite/c:tensorflowlite_c OK
bazel --output_user_root="…/output_user_root" build -c opt --cxxopt="–std=c++11" --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cpu=armeabi-v7a --config=android_arm //tensorflow/lite/java:tensorflowlite

bazel --output_user_root="…/output_user_root" build --cxxopt="–std=c++11" -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=armeabi-v7a --config=android_arm //tensorflow/lite/java:tensorflow-lite

D:/SoftWare/Android/Sdk/ndk/21.3.6528147

D:/SoftWare/Android/Sdk

D:/SoftWare/Android/Sdk/ndk/android-ndk-r18b
D:/SoftWare/Android/Sdk
D:/SoftWare/Android/Sdk/ndk/android-ndk-r12b

external/androidndk/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/…/…/…/…/arm-linux-androideabi/lib/…/lib/armv7-a/libatomic.a