机器学习 部署 嵌入式_如何在移动和嵌入式设备上部署机器学习模型

机器学习 部署 嵌入式

介绍 (Introduction)

Thanks to libraries such as Pandas, scikit-learn, and Matplotlib, it is relatively easy to start exploring datasets and make some first predictions using simple Machine Learning (ML) algorithms in Python. Although, to make these trained models useful in the real world, it is necessary to make them available to make predictions on either the Web or Portable devices.

多亏了Pandas，scikit-learn和Matplotlib等库，使用Python中的简单机器学习(ML)算法开始探索数据集并做出一些初步预测相对容易。 虽然，为了使这些训练有素的模型在现实世界中有用，有必要使它们可用于在Web或便携式设备上进行预测。

In two of my previous articles, I explained how to create and deploy a simple Machine Learning model using Heroku/Flask and Tensorflow.js. Today, I will instead explain to you how to deploy Machine Learning models on Smartphones and Embedded Devices using TensorFlow Lite.

在前两篇文章中，我解释了如何使用Heroku / Flask和Tensorflow.js创建和部署简单的机器学习模型。 今天，我将向您解释如何使用TensorFlow Lite在智能手机和嵌入式设备上部署机器学习模型。

TensorFlow Lite (TensorFlow Lite)

TensorFlow Lite is a platform developed by Google to train Machine Learning models on mobile, IoT (Interned of Things) and embedded devices.

TensorFlow Lite是Google开发的一个平台，用于在移动，IoT(物联网)和嵌入式设备上训练机器学习模型。

Using TensorFlow Lite, all the workflow is executed within the device, which avoids having to send data back and forth from a server. Some of the main advantages of doing this are:

使用TensorFlow Lite，所有工作流程都在设备内执行，从而避免了必须从服务器来回发送数据的情况。 这样做的一些主要优点是：

• Increased privacy since the data doesn't have to leave the device (this can allow you to apply techniques such as Differential Privacy and Federated Learning).

  由于数据不必离开设备，因此提高了隐私性(这可以使您应用差分隐私和联合学习之类的技术 )。

• Reduced power consumption, because no internet connection is required.
  降低了功耗，因为不需要互联网连接。
• Decreased latency, since there is no communication with the server.
  延迟减少，因为与服务器之间没有通信。

TensorFlow Lite offers API support for different languages such as Python, Java, Swift and C++.

TensorFlow Lite为不同的语言提供API支持，例如Python，Java，Swift和C ++。

A typical workflow using TensorFlow Lite would consist of:

使用TensorFlow Lite的典型工作流程包括：

1. Creating and training a Machine Learning model in Python using TensorFlow.
   使用TensorFlow在Python中创建和训练机器学习模型。

2. Converting our model in a suitable format for TensorFlow Lite using TensorFlow Lite converter.

   使用TensorFlow Lite转换器将模型转换为适用于TensorFlow Lite的格式。

3. Deploying our Machine Learning model on our mobile device using TensorFlow Lite interpreter.

   使用TensorFlow Lite解释器在移动设备上部署机器学习模型。

4. Optimising the model memory consumption and accuracy.
   优化模型的内存消耗和准确性。

There are several techniques which have been developed during the last few years in order to reduce the memory consumption of Machine Learning models [1]. One example is Model Quantization.

在过去的几年中，已经开发了几种技术来减少机器学习模型的内存消耗[1]。 一个例子是模型量化。

Model Quantization aims to reduce:

模型量化旨在减少：

1. The precision representations of Artificial Neural Networks weights (eg. converting 34.3456657 to 34.3).
   人工神经网络权重的精确表示(例如，将34.3456657转换为34.3)。
2. The memory access costs for reading and storing intermediate activation functions.
   用于读取和存储中间激活功能的内存访问成本。

Using Model Quantization can therefore lead to a reduction in the model latency and size. One of the main drawbacks of this technique is a slight decrease in accuracy (which can be more or less important depending on the applications).

因此，使用模型量化可以减少模型等待时间和减小大小。 该技术的主要缺点之一是精度略有下降(取决于应用，它可能或多或少重要)。

According to the TensorFlow Lite documentation, taking the Inception_v3 Image Classifier as example, using Model Quantization can lead to up to 0.8% decrease in accuracy. On the other hand, using Model Quantization made it possible to reduce the model size by 4 times (95.7MB vs 23.9MB) and the latency by 285ms (1130ms vs 845ms) [2].

根据TensorFlow Lite文档，以Inception_v3图像分类器为例，使用模型量化可能导致精度降低多达0.8％。 另一方面，使用模型量化可以将模型大小减少4倍(95.7MB和23.9MB)，并将延迟减少285ms(1130ms和845ms)[2]。

In the following example, I will  demonstrate how to use a pre-trained model in an Android Application.

在以下示例中，我将演示如何在Android应用程序中使用预训练的模型。

示范 (Demonstration)

As a practical example, I recently created an Android Studio App using the pre-trained Inception v3 Image Classifier thanks to TensorFlow Lite.

作为一个实际示例，由于TensorFlow Lite，我最近使用了经过预训练的Inception v3图像分类器创建了一个Android Studio应用程序。

盗梦空间v3 (Inception v3)

The Inception Classifier was created in order to solve some limitations brought from creating very large and deep neural networks for image classification tasks.

创建初始分类器是为了解决为图像分类任务创建超大型深度神经网络带来的一些限制。

A few years ago, in order to solve image classification tasks, were created deep learning models composed by an always increasing number of layers and neurons in each layer. Taking this approach, it was possible to score better results, but it also also led to two main problems:

几年前，为了解决图像分类任务，创建了深度学习模型，该模型由数量不断增加的层和每一层中的神经元组成。 采用这种方法，可能会获得更好的结果，但同时也导致两个主要问题：

1. Increasing amount of computational power required to train our model.
   训练模型所需的计算能力不断增加。
2. Increasing probability of overfitting (making our model perform really well during training stage but not so much when working with brand new data).
   过度拟合的可能性增加(使我们的模型在训练阶段确实表现良好，但在处理全新数据时却没有那么多)。

The creation of the Inception Classifier aimed instead to solve these problems by heavily engineering the characteristics of the model, increasing the training speed. During the last few years, different versions of the Inception classifier have been created. In the following example I decided to use the v3 version.

相反，创建Inception分类器的目的是通过大量设计模型的特征来提高训练速度，从而解决这些问题。 在最近几年中，创建了不同版本的Inception分类器。 在以下示例中，我决定使用v3版本。

Android图像分类器 (Android Image Classifier)

In order to create this App, I decided to use Android Studio as the IDE. In this way, it was relatively easy to integrate all the required TensorFlow Lite dependencies to load the Inception v3 model.

为了创建此应用程序，我决定使用Android Studio作为IDE。 通过这种方式，集成所有必需的TensorFlow Lite依赖项以加载Inception v3模型相对容易。

I successively decided to divide this App in two main windows:

我先后决定将该应用程序划分为两个主要窗口：

• In the first one, the user is welcomed and is asked to select the image classifier to use (in this case the "Inception Quantized Classifier"). Then, the user is asked to take a picture of the object they want to classify and finally the picture is cropped. In this case, it was necessary to crop the image since the Inception Classifier takes just squared images as inputs.
  在第一个中，欢迎用户并要求其选择要使用的图像分类器(在这种情况下为“初始量化分类器”)。 然后，要求用户为他们要分类的对象拍照，最后裁剪照片。 在这种情况下，由于Inception分类器仅将平方的图像作为输入，因此有必要裁剪图像。
• In the second window, the user is finally ask to click on the "Classify Image" button in order to get in return the 3 most likely predictions made by the classifier.
  在第二个窗口中，用户最终被要求单击“分类图像”按钮，以获取分类器做出的3个最可能的预测。

In order to realise this project, I referenced Michael Shea and the TensorFlow Lite documentation implementations [3, 4].

为了实现此项目，我引用了Michael Shea和TensorFlow Lite文档实现[3，4]。

If you are interested in testing out yourself this App, all the code and an Apk of the App is available in my Github repository.

如果您有兴趣测试自己该应用程序，则我的Github存储库中提供了该应用程序的所有代码和Apk。

In case you are planning to start your own project using TensorFlow Lite, their documentation is probably to best place where to start.

如果您打算使用TensorFlow Lite启动自己的项目，那么他们的文档可能是最好的起点。

联络人 (Contacts)

If you want to keep updated with my latest articles and projects follow me and subscribe to my mailing list. These are some of my contacts details:

如果您想随时了解我的最新文章和项目，请关注我并订阅我的邮件列表 。 这些是我的一些联系方式：

• Linkedin

  领英

• Personal Blog

  个人博客

• Personal Website

  个人网站

• Medium Profile

  中档

• GitHub

  的GitHub

• Kaggle

  卡格勒

参考书目 (Bibliography)

[1] Nimit S. Sohoni, Christopher R. Aberger et al, Low-Memory Neural Network Training: A Technical Report. Accessed at: https://arxiv.org/pdf/1904.10631.pdf

[1] Nimit S. Sohoni，Christopher R. Aberger等人，《低内存神经网络训练：技术报告》。 访问： https ： //arxiv.org/pdf/1904.10631.pdf

[2] Model optimization, TensorFlow. Accessed at: https://www.tensorflow.org/lite/performance/model_optimization

[2]模型优化，TensorFlow。 访问以下网址 ： https ： //www.tensorflow.org/lite/performance/model_optimization

[3] Michael Shea, TensorFlow Lite Inception Model Android Tutorial. Accessed at: https://www.youtube.com/watch?v=8zQsAl2z4iU

[3] Michael Shea，TensorFlow Lite初始模型Android教程。 访问网址 ： https : //www.youtube.com/watch?v=8zQsAl2z4iU

[4] TensorFlow Lite Documentation, TensorFlow Lite image classification Android example application. Accessed at: https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android

[4] TensorFlow Lite文档，TensorFlow Lite图像分类Android示例应用程序。 访问以下网址 ： https ： //github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android

翻译自: https://www.freecodecamp.org/news/machine-learning-for-mobile-and-embedded-devices/

机器学习 部署 嵌入式