Bazel编译tensorflow C++及批量识别问题

目标：在windows环境下实现c++批量识别图片

方法一：用Tensorflow的C++接口

以下是一些参考，但是没有起到实质性的帮助：

Using a batchsize greater than 1 when using C++ API
https://github.com/tensorflow/tensorflow/issues/3362
Read the TF FAQ, you need export / freeze the graph without specifying the batch size.

how-to-load-trained-tensorflow-model-to-predict-with-different-batch-size
https://stackoverflow.com/questions/49872844/how-to-load-trained-tensorflow-model-to-predict-with-different-batch-size?rq=1

what-is-the-best-way-to-run-saved-model-with-different-batch-size-in-tensorflow

batch_size_tensor = tf.placeholder_with_default(128, shape=[])
input_tensors = tf.train.batch(..., batch_size=batch_size_tensor, ...)

how-to-concat-multi-tensors-in-tensorflow-c-framework
https://stackoverflow.com/questions/49746501/how-to-concat-multi-tensors-in-tensorflow-c-framework

一篇有用的教程，的确可以在windows上重新编译tensorflow。
How To Build a Tensorflow on Windows from source code with CMake - Visual Studio 2017 (2015 platform toolset) Cuda 8 Cudnn 6
http://anonimousindonesian.blogspot.com/2017/12/tutorial-how-to-build-tensorflow-on_23.html

Tensorflow C++ API Batch Inference的一些问题：
https://github.com/peter6888/tf_opencv
https://medium.com/greppy/object-detection-using-a-ssd-mobilenet-coco-model-with-opencv-3-3-tensorflow-1-4-in-c-and-xcode-28b3e1d955db
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/c/c_api_experimental_test.cc
https://medium.com/@fanzongshaoxing/use-tensorflow-c-api-with-opencv3-bacb83ca5683
http://anonimousindonesian.blogspot.com/2017/12/tutorial-how-to-build-tensorflow-on_23.html
https://stackoverflow.com/questions/44790456/how-to-feed-inputs-into-a-loaded-tensorflow-model-using-c
https://github.com/tensorflow/tensorflow/issues/19909
https://github.com/memo/ofxMSATensorFlow/issues/34

【Exactly my problem】
https://github.com/tensorflow/tensorflow/issues/19909

【No use】
https://www.mabl.com/blog/image-classification-with-tensorflow

方法二：通过一些封装好的DLL实现

deep-learning-library-10-fast-neural-network
https://baptiste-wicht.com/posts/2017/10/deep-learning-library-10-fast-neural-network-library.html
https://github.com/wichtounet/dll

方法三：利用Visual Studio的AL工具

Python和C++的整合
Visual Studio Tools for AI
https://github.com/Microsoft/vs-tools-for-ai/tree/master/docs

C++的批量预测问题
Microsoft Windows 10 VS 2015
Known-good configurations
Microsoft Windows 10
Microsoft Visual Studio Enterprise 2015 with Visual C++ 2015
Anaconda 4.1.1 (Python 3.5 64-bit)
Git for Windows version 2.9.2.windows.1
swigwin-3.0.10
[NVidia CUDA Toolkit 8.0] (https://developer.nvidia.com/cuda-downloads)
[NVidia CUDNN 5.1] (https://developer.nvidia.com/cudnn)
CMake 3.6
https://github.com/tensorflow/tensorflow/tree/395cfc42ee3c5842f5383f4049674c012998b133/tensorflow/contrib/cmake

https://github.com/cjweeks/tensorflow-cmake
https://github.com/HamedMP/tensorflow_export_cpp_example

【跑通】
https://github.com/hemajun815/tensorflow_cc_image_classify
Load images from disk and train a classifier with TensorFlow_cc.
Up to now, the fc model is working, but the cnn model is under maintenance.

方法四：用Keras训练，训练好的模型用TF的C++接口预测

【教程】Run your Keras models in C++ Tensorflow
http://www.bitbionic.com/2017/08/18/run-your-keras-models-in-c-tensorflow/

[python3的虚拟环境]
https://blog.csdn.net/qingche456/article/details/65465760

https://github.com/tensorflow/tensorflow

[Installing Bazel]
https://docs.bazel.build/versions/master/install.html

[Problem 1]
output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None
[Solution 1]
https://stackoverflow.com/questions/44149093/valueerror-output-of-generator-should-be-a-tuple-x-y-sample-weight-or-x?rq=1

[Problem 2]
The TensorFlow library wasn’t compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
[Solution 2]

pip install tensorflow-gpu==1.1.0

[Problem 3]
save_model requires h5py
[Solution 3]

pip install --upgrade h5py

python k2tf_trainer.py --test=../../data/flowers/raw-data/validation --train=../../data/flowers/raw-data/train --cats=5 --shape=80 --batch=1024 --epochs=400 --output=./temp

–epochs=400 does not work !! I don’t know why

python k2tf_eval.py -m '/home/bitwise/Development/keras-to-tensorflow/temp/k2tf-20170816140235/e-075-vl-0.481-va-0.837.h5' -i '/home/bitwise/data/flowers/raw-data/validation/tulips/450607536_4fd9f5d17c_m.jpg' -s 80

Exception ignored in: del of
>
Traceback (most recent call last): File
“/home/dir/py3env/lib/python3.5/site-packages/tensorflow/python/client/session.py”,
line 696, in del File
“/home/dir/py3env/lib/python3.5/site-packages/tensorflow/python/framework/c_api_util.py”,
line 30, in init TypeError: ‘NoneType’ object is not callable

h5转pb

remove the orginal one first

python k2tf_convert.py -m '/home/bitwise/Development/keras-to-tensorflow/temp/k2tf-20170816140235/e-075-vl-0.481-va-0.837.h5' -n 1

Output nodes names are: [‘k2tfout_0’] Saved the graph definition in
ascii format at: ./graph_def_for_reference.pb.ascii Converted 10
variables to const ops. Saved the constant graph (ready for inference)
at: ./output_graph.pb

Running your Tensorflow model with Python

python label_image.py --graph=./output_graph.pb --labels=../../data/flowers/raw-data/labels.txt --input_width=80 --input_height=80 --input_layer=firstConv2D_input --output_layer=k2tfout_0 --image=../../data/flowers/raw-data/validation/dandelion/13920113_f03e867ea7_m.jpg 
Running your Tensorflow model with C++
~/Development/keras-to-tensorflow $ git clone https://github.com/tensorflow/tensorflow
~/Development/keras-to-tensorflow $ cd tensorflow/
~/Development/keras-to-tensorflow/tensorflow $ git checkout r1.4

sudo apt-get install build-essential python3-numpy python3-dev python3-pip python3-wheel

sudo apt-get install openjdk-8-jdk

NOTE: I have not tested building with CUDA – this is just the
documentation that I’ve read. For deployment I didn’t want to build
with CUDA, however if you do then you of course need the CUDA SDK and
the CUDNN code from NVIDIA. You’ll also need to grab libcupti-dev.

echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
sudo apt-get update && sudo apt-get install bazel
sudo apt-get upgrade bazel

[Problem]
W: GPG 错误：http://storage.googleapis.com/bazel-apt stable InRelease: 由于没有公钥，无法验证下列签名： NO_PUBKEY 3D5919B448457EE0
W: 仓库 “http://storage.googleapis.com/bazel-apt stable InRelease” 没有数字签名。
N: 无法认证来自该源的数据，所以使用它会带来潜在风险。
N: 参见 apt-secure(8) 手册以了解仓库创建和用户配置方面的细节。

[Solution]
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3D5919B448457EE0

[Problem]

ERROR: /home/dir/Development/youtube-tutorial/tensorflow-r1.4/WORKSPACE:15:1: Traceback (most recent call last):
	File "/home/dir/Development/youtube-tutorial/tensorflow-r1.4/WORKSPACE", line 15
		closure_repositories()
	File "/home/dir/.cache/bazel/_bazel_dir/d49c392071b5141d1a40f1929ac03f7b/external/io_bazel_rules_closure/closure/repositories.bzl", line 69, in closure_repositories
		_check_bazel_version("Closure Rules", "0.4.5")
	File "/home/dir/.cache/bazel/_bazel_dir/d49c392071b5141d1a40f1929ac03f7b/external/io_bazel_rules_closure/closure/repositories.bzl", line 172, in _check_bazel_version
		fail(("%s requires Bazel >=%s but was...)))
Closure Rules requires Bazel >=0.4.5 but was 0.16.1

[Solution]
安装Bazel 0.5.4
[Problem]

/usr/bin/ld: warning: libcublas.so.8.0, needed by ./tensorflow/bazel-bin/tensorflow/libtensorflow_cc.so, not found 
libtensorflow_framework.so: error adding symbols: DSO missing from command line
/usr/bin/ld: main.o: undefined reference to symbol '_ZN10tensorflow6TensorC1ENS_8DataTypeERKNS_11TensorShapeE'
libtensorflow_framework.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
/usr/bin/ld: warning: libcublas.so.8.0, needed by ./tensorflow/bazel-bin/tensorflow/libtensorflow_cc.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libcusolver.so.8.0, needed by ./tensorflow/bazel-bin/tensorflow/libtensorflow_cc.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libcudart.so.8.0, needed by ./tensorflow/bazel-bin/tensorflow/libtensorflow_cc.so, not found (try using -rpath or -rpath-link)
main.o：在函数‘ReadEntireFile(tensorflow::Env*, std::string const&, tensorflow::Tensor*)’中：

[Solution]

sudo ln -sf libcudnn.so.6.0.21 libcudnn.so.6
sudo ln -sf libcudnn.so.6 libcudnn.so
sudo ldconfig

[Problem]
./tensorflow/bazel-genfiles/tensorflow/core/framework/tensor.pb.h:23:35: fatal error: google/protobuf/arena.h: 没有那个文件或目录
#include
https://github.com/BVLC/caffe/issues/4988

I solved it (on Ubuntu 16.04) by:

sudo apt-get remove libprotobuf-dev protobuf-compiler
sudo apt-get install libprotobuf-dev protobuf-compiler

https://github.com/tensorflow/tensorflow/issues/14620

TF_INC=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')
TF_LIB=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())')
g++ -std=c++11 -shared zero_out.cc -o zero_out.so -fPIC -I$TF_INC -I$TF_INC/external/nsync/public -L$TF_LIB -ltensorflow_framework -O2

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler 
sudo apt-get install --no-install-recommends libboost-all-dev

https://github.com/protocolbuffers/protobuf/blob/master/src/README.md
http://wangkejie.me/2018/03/01/tensorflow-cplusplus-installation/
https://github.com/cjweeks/tensorflow-cmake
http://answers.opencv.org/question/177644/weird-behaviour-when-linking-libtensorflow_framework14-with-opencv/
https://www.tensorflow.org/install/install_sources
https://github.com/kaldi-asr/kaldi/issues/1898
https://github.com/tensorflow/tensorflow/issues/14632

./configure

bazel build --config=monolithic

bazel build  --config=opt --config=cuda tensorflow/tools/pip_package:build_pip_package --action_env="LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" 

bazel build --jobs=6 --verbose_failures -c //tensorflow:libtensorflow_cc.so

bazel build -c opt --config=cuda --copt=-march=native //tensorflow:libtensorflow_cc.so

bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer

[Problem]

warning: libtensorflow_framework.so, needed by
./tensorflow/bazel-bin/tensorflow/libtensorflow_cc.so, not found (try
using -rpath or -rpath-link)

/usr/bin/ld: warning: libtensorflow_framework.so, needed by
./tensorflow/bazel-bin/tensorflow/libtensorflow_cc.so, not found (try
using -rpath or -rpath-link)

[Problem]

WARNING:
/home/dir/Development/youtube-tutorial/tensorflow/tensorflow/core/BUILD:1781:1:
in includes attribute of cc_library rule
//tensorflow/core:framework_headers_lib: ‘…/…/external/nsync/public’
resolves to ‘external/nsync/public’ not below the relative path of its
package ‘tensorflow/core’. This will be an error in the future. Since
this rule was created by the macro ‘cc_header_only_library’, the error
might have been caused by the macro implementation in
/home/dir/Development/youtube-tutorial/tensorflow/tensorflow/tensorflow.bzl:1047:30

[Solution 1]
[ref] https://github.com/tensorflow/tensorflow/issues/12482

/* Copyright 2015 The TensorFlow Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/

#ifndef TENSORFLOW_PLATFORM_DEFAULT_MUTEX_H_
#define TENSORFLOW_PLATFORM_DEFAULT_MUTEX_H_

// IWYU pragma: private, include "third_party/tensorflow/core/platform/mutex.h"
// IWYU pragma: friend third_party/tensorflow/core/platform/mutex.h

#include <chrono>
#include <condition_variable>
#include <mutex>
#include "tensorflow/core/platform/thread_annotations.h"
namespace tensorflow {

#undef mutex_lock

enum LinkerInitialized { LINKER_INITIALIZED };

// A class that wraps around the std::mutex implementation, only adding an
// additional LinkerInitialized constructor interface.
class LOCKABLE mutex : public std::mutex {
 public:
  mutex() {}
  // The default implementation of std::mutex is safe to use after the linker
  // initializations
  explicit mutex(LinkerInitialized x) {}

  void lock() ACQUIRE() { std::mutex::lock(); }
  bool try_lock() EXCLUSIVE_TRYLOCK_FUNCTION(true) {
    return std::mutex::try_lock();
  };
  void unlock() RELEASE() { std::mutex::unlock(); }
};

class SCOPED_LOCKABLE mutex_lock : public std::unique_lock<std::mutex> {
 public:
  mutex_lock(class mutex& m) ACQUIRE(m) : std::unique_lock<std::mutex>(m) {}
  mutex_lock(class mutex& m, std::try_to_lock_t t) ACQUIRE(m)
      : std::unique_lock<std::mutex>(m, t) {}
  mutex_lock(mutex_lock&& ml) noexcept
      : std::unique_lock<std::mutex>(std::move(ml)) {}
  ~mutex_lock() RELEASE() {}
};

// Catch bug where variable name is omitted, e.g. mutex_lock (mu);
#define mutex_lock(x) static_assert(0, "mutex_lock_decl_missing_var_name");

using std::condition_variable;

inline ConditionResult WaitForMilliseconds(mutex_lock* mu,
                                           condition_variable* cv, int64 ms) {
  std::cv_status s = cv->wait_for(*mu, std::chrono::milliseconds(ms));
  return (s == std::cv_status::timeout) ? kCond_Timeout : kCond_MaybeNotified;
}

}  // namespace tensorflow

#endif  // TENSORFLOW_PLATFORM_DEFAULT_MUTEX_H_

[Solution2]

[gcc/g++ 链接库的编译与链接]
https://blog.csdn.net/q_l_s/article/details/51313842

https://blog.csdn.net/lovebyz/article/details/80620851

TensorFlow的C++接口
https://www.tensorflow.org/install/install_c
https://blog.csdn.net/akadiao/article/details/81451505

keras2cpp
https://blog.csdn.net/qq_33039859/article/details/80098073
https://github.com/pplonski/keras2cpp
图片: https://images-cdn.shimo.im/eQ4P02mz82wIw3W1/image.png

DLL

现在有些操作在GPU上实现时并不高效，例如Batch Normalization and
Dropout。我想保证所有的操作都可以在GPU上进行有效的计算。还有一些在CPU上效率不高的情况，例如，批处理标准化目前很差。一些SGD优化器，如Nadam也很慢。

http://www.cnhonkerarmy.com/forum.php?mod=viewthread&tid=223688
https://github.com/amir-abdi/keras_to_tensorflow

其它参考资料：
Keras训练模型的C++调用尝试
https://blog.csdn.net/akadiao/article/details/81451505
C++从零实现深度神经网络之六——实战手写数字识别（sigmoid和tanh）
https://blog.csdn.net/xingchenbingbuyu/article/details/53736154

The CUDA optimizations would internally be used for C++ functions so it doesn’t make much of a difference with Python + OpenCV.

有错误或者需要更正的地方，欢迎补充?

吴恩达
https://blog.csdn.net/u013733326/article/details/79971488
https://download.csdn.net/download/daizongxue/10024426

直方图代码
https://blog.csdn.net/bjbz_cxy/article/details/79716317

直方图结果
https://shimo.im/docs/DZ5FU18PXpMVc0d7/

64*64手势识别作为参考
https://blog.csdn.net/qq_31823267/article/details/78239870
https://www.jianshu.com/p/8833c7a6a57c?utm_source=oschina-app