机器学习 量化 滚动训练_机器学习和深度学习模型的训练后分析和量化

机器学习 量化 滚动训练

虚拟援助项目 (VIRTUAL ASSISTANT PROJECT)

Building and exploring machine learning as well as deep learning models is something I love to do. I pick a project and start working on it. Some are just hobbyist level projects and some are projects with real-use cases and advantages. I have a moderate GPU and a moderate system with decent processing capabilities. So all I focus on is just training and making sure the model works and runs on my PC. However, this is not the end of the story and sometimes I end up missing out on an extremely important concept — Post-training analysis of the model.

乙 uilding并探索机器学习以及深学习模型是我喜欢做的事。 我选择一个项目并开始研究。 有些只是爱好者级别的项目，有些是具有实际用例和优势的项目。 我有一个中等的GPU和一个具有不错处理能力的中等系统。 因此，我只关注培训并确保模型能够在PC上正常运行。 但是，这还不是故事的结局，有时我最终会错过一个非常重要的概念-模型的训练后分析 。

What is this post-training analysis and why is it so important?

培训后的分析是什么？为何如此重要？

Post-training analysis sometimes also referred to as post-mortem analysis plays a major role in the optimization of models. The business models built and trained need to be optimized in order for them to work efficiently on lower-end devices and embedded systems, like the raspberry pi. One of the principal components of building and evaluating models is examining the predictive capabilities and performance quality of the model. A more paramount concept is understanding the limitations of your machine learning or deep learning model. Overcoming these limitations is the key to a successful model.

训练后分析有时也称为事后分析在模型优化中起主要作用。 需要优化构建和培训的业务模型，以使其在低端设备和嵌入式系统(如树莓派)上有效地工作。 建立和评估模型的主要组成部分之一是检查模型的预测能力和性能质量。 一个更重要的概念是了解机器学习或深度学习模型的局限性。 克服这些限制是成功建立模型的关键。

In this article, we will discuss how to quantize our models effectively for particular use cases. We will be focusing mainly on the discussion of the limitations, improvements, and quantization of deep learning models. Firstly, we will discuss what exactly the post-training quantization of our models using TensorFlow is. We will learn how to make our models more dynamic and effective across all platforms to reach a wider target audience with the help of various optimization techniques. Then we will understand limitations and techniques which can be used for the improvement of a variety of models. Our main references for this project will be the TensorFlow blog for post-training quantization and the virtual assistant project series which was done by me. You can check out both of these in the links provided below albeit not mandatory.

在本文中，我们将讨论如何针对特定用例有效地量化模型。 我们将主要专注于深度学习模型的局限性，改进和量化的讨论。 首先，我们将讨论使用TensorFlow对我们的模型进行精确的训练后量化。 我们将学习如何借助各种优化技术使我们的模型在所有平台上更具动态性和有效性，以覆盖更广泛的目标受众。 然后，我们将了解可用于改进各种模型的局限性和技术。 我们对该项目的主要参考将是TensorFlow博客(用于训练后量化)和由我完成的虚拟助手项目系列。 您可以在下面提供的链接中查看这两个字段，尽管不是必需的。

Note: Although not an essential requirement, I would highly recommend viewers to check out my previous projects on the smart face lock system, next word prediction, and the innovative chatbot using 1-D convolution layers projects from the virtual assistant series as it would help in grasping some of the concepts in a more intuitive manner.

注意：虽然不是必需的要求，但我还是强烈建议观众查看我以前在智能面部锁定系统上的项目，下一词预测以及使用虚拟助手系列中的一维卷积层项目的创新聊天机器人，因为这样做会有所帮助以更直观的方式掌握一些概念。

Photo by Michael Dziedzic on Unsplash Michael Dziedzic在 Unsplash上 拍摄的照片

训练后量化： (Post-Training Quantization:)

A model that runs effectively on your system might not be able to run the same program/model effectively on a lower-end device. This could be due to the hardware constraints of the target device. Here, is where post-training quantization can help improve in the optimization of the algorithms and models for the target device. Post-training quantization is a conversion technique that can reduce model size while also improving CPU and hardware accelerator latency, with little degradation in model accuracy. You can quantize an already trained float TensorFlow model when you convert it to the TensorFlow Lite format using the TensorFlow Lite Converter. The TensorFlow Lite Converter is very useful on devices such as the Raspberry Pi for optimization of object detection models, face recognition models, etc. The object detection projects can be optimized using TensorFlow Lite up to a great effect on android or ios devices as well. You can check out this cool project for this topic here.

在您的系统上有效运行的模型可能无法在低端设备上有效运行同一程序/模型。 这可能是由于目标设备的硬件限制。 在这里，训练后量化可以帮助改善目标设备的算法和模型的优化。 训练后量化是一种转换技术，可以减小模型大小，同时还可以改善CPU和硬件加速器的延迟，而模型精度几乎不会降低。 使用TensorFlow Lite转换器将已训练的float TensorFlow模型转换为TensorFlow Lite格式时，您可以对其进行量化。 TensorFlow Lite Converter在诸如Raspberry Pi之类的设备上非常有用，可优化对象检测模型，面部识别模型等。使用TensorFlow Lite可以优化对象检测项目，同时在android或ios设备上也能发挥很大的作用。 您可以在这里查看这个主题的出色项目。

Below is the block diagram to help you choose the best post-training quantization and optimization methods for your model:

以下是框图，可帮助您为模型选择最佳的训练后量化和优化方法：

Source: Image from tensorflow 来源：来自tensorflow的图像

In brief, the various post-training quantization methods are as follows:

简而言之，各种训练后量化方法如下：

动态范围量化 (Dynamic range quantization)

This simplest form of post-training quantization. It statically quantizes only the weights from floating point to integer, which has 8-bits of precision.

训练后量化的最简单形式。 它仅静态量化从浮点到整数的权重，该精度具有8位精度。

全整数量化 (Full integer quantization)

You can get further latency improvements, reductions in peak memory usage, and compatibility with integer only hardware devices or accelerators by making sure all model math is integer quantized.

通过确保所有模型数学都是整数量化的，您可以得到进一步的延迟改进，峰值内存使用量的减少以及与仅整数的硬件设备或加速器的兼容性。

Float16量化 (Float16 quantization)

You can reduce the size of a floating point model by quantizing the weights to float16, the IEEE standard for 16-bit floating point numbers.

您可以通过将权重量化为float16(16位浮点数的IEEE标准)来减小浮点模型的大小。

Basically, in these 3 methods of post-training optimization, we are making sure that our model design is as efficient and optimized as possible. We have to make sure that once the necessary optimization changes are done, the model remains performance efficient while maintaining more or less, the same effectiveness in parameters such as accuracy as well as loss.

基本上，在这三种训练后优化方法中，我们要确保模型设计尽可能高效和优化。 我们必须确保一旦完成必要的优化更改，模型就可以保持性能高效，同时保持或多或少的参数有效性(如准确性和损失)相同。

Photo by Dhru J on Unsplash Dhru J在 Unsplash上的 照片

用于理解模型局限性以及可以进行的重大更改和改进的其他直观示例- (Other Intuitive Examples for understanding the limitations of models and significant changes and improvements that can be made —)

1.人脸识别模型： (1. Face Recognition Models:)

So, you successfully built a face recognition model using deep learning techniques like transfer learning similar to the one here. The overall model performs extremely well on the trained data. Upon testing on the training and validation data, the results produced are still very good. The model performs as desired in most scenarios. However, there are some limitations about the model. Let us understand and look at these limitations of the face recognition model. Let us also look at the improvements that can be made on the following.

因此，您使用类似于此处的转移学习之类的深度学习技术成功构建了人脸识别模型。 总体模型在训练后的数据上表现非常出色。 通过对训练和验证数据的测试，得出的结果仍然非常好。 该模型在大多数情况下均按预期执行。 但是，该模型存在一些限制。 让我们了解并查看人脸识别模型的这些局限性。 让我们还看一下可以在以下方面进行的改进。

局限性： (Limitations:)

1. The face recognition model cannot perform very well with a low-quality camera. The camera quality must at least be average to capture the real-time face correctly and grant access.
   脸部识别模型在使用低画质相机时无法表现良好。 相机质量必须至少达到平均水平，才能正确捕获实时面部并授予访问权限。
2. The surrounding lighting must not be dark. It needs to be at least moderately bright. Otherwise, the model will have issues detecting and recognizing the face during real-time analysis.
   周围的灯光不能太暗。 它必须至少是中等亮度。 否则，模型将在实时分析过程中检测和识别人脸时出现问题。
3. We make use of the haarcascade_frontalface_default.xml for the detection of the faces. This can be problematic for identifying the faces at certain angles and the detection of the face might not work as desired.
   我们利用haarcascade_frontalface_default.xml来检测人脸。 这对于以特定角度识别面部可能是有问题的，并且面部的检测可能无法按预期进行。

改进之处： (Improvements:)

1. One-shot learning and training methods can be used for reducing the training time for each face. Since the current model recognizes only one face, if we want to add more faces we need to re-train the entire model. For this reason methods like one-shot learning needs to be considered for improving the quality and performance of the models.
   一次性学习和训练方法可用于减少每张脸的训练时间。 由于当前模型只能识别一张脸，因此，如果要添加更多的脸，则需要重新训练整个模型。 因此，需要考虑使用像一次性学习之类的方法来提高模型的质量和性能。
2. Alternatives to haarcascade_frontalface_default.xml can be found to improve the accuracy of the detection of the faces at any particular angle. An alternative can be to make a custom XML file for both front and side faces.
   可以找到haarcascade_frontalface_default.xml的替代方法，以提高在任何特定角度进行面部检测的准确性。 另一种选择是为正面和侧面制作自定义XML文件。
3. To make it run on embedded devices changes can be made on the memory constraints like converting to tf.float(32) and also changes can be made on the model by considering the use of tflite.
   要使其在嵌入式设备上运行，可以在内存限制方面进行更改，例如转换为tf.float(32)，还可以通过考虑使用tflite在模型上进行更改。

2.下一个单词预测： (2. Next Word Prediction:)

The next word prediction model is able to achieve a decent loss on the model. The next word model is a predictive search type approach which can be used for google searches. These can be also used to detect certain patterns of users and can be used in email and texting next word prediction tasks. The model despite being decent has flaws and there are ways to improve the flaws that exist on the next word prediction model. Let us look at all the limitations and the changes that can be made to improve the overall model.

下一个单词预测模型能够在该模型上实现不错的损失。 下一个单词模型是可用于Google搜索的预测搜索类型方法。 这些还可以用于检测特定的用户模式，并可以用于电子邮件和发短信预测下一个单词的任务。 该模型尽管很不错，但仍有缺陷，并且有一些方法可以改善下一个单词预测模型中存在的缺陷。 让我们看一下所有的局限性和可以为改善整体模型而进行的更改。

局限性： (Limitations:)

1. The next word prediction is limited to only a few words and cannot perform very accurately on longer sentences. This is not the best scenario while working in the industry as there might be longer sentences.
   下一个单词的预测仅限于几个单词，不能在较长的句子上非常准确地执行。 这不是在行业中工作的最佳方案，因为可能要更长的刑期。
2. The model requires a longer time to be trained and perform well compared to other models due to the use of LSTM layers. This longer training may be a limitation in the case of smaller embedded systems with lesser ram and GPU capabilities. However, once trained on a better system the deployment is not such a big issue.
   由于使用了LSTM层，因此与其他模型相比，该模型需要更长的时间来训练并表现良好。 在具有较小的内存和GPU功能的较小的嵌入式系统的情况下，这种较长的训练可能是一个限制。 但是，一旦接受了更好的系统培训，部署就不是一个大问题。

改进之处： (Improvements:)

1. Zero-shot and one-shot learning methods even exist for natural language processing. The same methods can be used for better training of the model to improve the overall performance and avoid repeated training procedures which can be a really big hindrance in some real-life applications and scenarios. Hence, one-shot learning is a great alternative for deployment and working in other embedded systems with lower training capacities.
   零镜头和单镜头学习方法甚至可以用于自然语言处理。 可以使用相同的方法对模型进行更好的训练，以提高整体性能并避免重复训练过程，这在某些实际应用和场景中可能是一个很大的障碍。 因此，一次性学习是在培训能力较低的其他嵌入式系统中部署和工作的绝佳选择。
2. Advanced training algorithms like the GPT-3 model could very well be extremely effective for these predictive tasks.
   像GPT-3模型这样的高级培训算法对于这些预测性任务可能非常有效。
3. More methods of pre-processing can be used and many more alternative methods can be tested out to gain better accuracies, improved losses, and achieve overall high performance.
   可以使用更多的预处理方法，并且可以测试出更多的替代方法，以获得更好的准确性，改善的损耗并实现总体高性能。

3.聊天机器人模型： (3. Chatbot Models:)

The chatbot model is a text classification based innovative chatbot which can achieve an overall good accuracy and a reduced loss. The chatbot model is able to perform really well on the entire dataset which we used. However, even our chatbot has certain limitations and issues. We will cover them in the below section. But the overall prediction system and the chatbot built on the conv-1D architecture performs well on the witty.txt dataset and can be adopted for similar datasets. The model works on the classification of unique values to most frequently repeated questions.

聊天机器人模型是基于文本分类的创新聊天机器人，可以实现总体良好的准确性和减少的损失。 chatbot模型能够在我们使用的整个数据集上表现出色。 但是，即使我们的聊天机器人也有某些限制和问题。 我们将在以下部分中介绍它们。 但是，基于conv-1D体系结构的总体预测系统和聊天机器人在witty.txt数据集上表现良好，并且可以用于类似的数据集。 该模型适用于最常见重复问题的唯一值分类。

局限性： (Limitations:)

1. One of the main limitations of the conv-1D chatbot model is that it is a classification based chatbot and not a generative approach to modeling. This classification approach can lead to major setbacks as there might be issues in certain languages, like semantic issues when the user is not very proficient in the English language. This can be misleading and the chatbot might not produce the desired result.
   conv-1D聊天机器人模型的主要限制之一是它是基于分类的聊天机器人，而不是生成模型的方法。 这种分类方法可能会导致重大挫折，因为某些语言可能会出现问题，例如当用户不太精通英语时会出现语义问题。 这可能会产生误导，并且聊天机器人可能无法产生所需的结果。
2. Since it is not a completely perfect model, it can be error-prone sometimes. It can predict results that can not be satisfactory to the user and this might be a concern for industrial uses.
   由于它不是一个完美的模型，因此有时可能容易出错。 它可以预测用户无法满意的结果，这可能是工业用途的问题。

改进之处： (Improvements:)

1. More methods of pre-processing and natural language processing can be used to achieve a higher accuracy and reduced loss on the training of the model. This can also improve the overall predictions of the model.
   可以使用更多的预处理和自然语言处理方法来实现更高的准确性并减少模型训练的损失。 这也可以改善模型的整体预测。
2. Advanced training algorithms like the GPT-3 model works fantastic even for conversational chatbots and it is a great alternative method to train high-quality chatbots.
   像GPT-3模型这样的高级培训算法，即使对于对话型聊天机器人也非常有效，它是训练高质量聊天机器人的绝佳替代方法。
3. Other methods like transfer learning based classification, sequence to sequence models with attention, or even certain one-shot learning methods for training can be used.
   可以使用其他方法，例如基于转移学习的分类，具有注意力的序列到序列模型，甚至某些用于训练的单次学习方法。

Photo by William Iven on Unsplash 威廉· 伊文 ( William Iven)在 Unsplash上 摄

结论： (Conclusion:)

Having fun alongside experimenting with machine learning and deep learning models is always cool and the best part about them. While exploring these models, if you do want them to be converted to real-life use cases where it can reach as well as benefit a large number of people, then post-training analysis and post-training quantization of models becomes extremely important to improve the efficiency, quality, and compactness to deploy the projects to a wider audience. The post-training quantization also enables us to achieve almost identical accuracy on the quantized model similar to the accuracy obtained in the original model. This simplifies and makes our life a lot easier!

在尝试机器学习和深度学习模型的同时获得乐趣总是很酷的事情，也是与它们有关的最好的部分。 在探索这些模型时，如果您希望将它们转换为可以触及并惠及大量人的实际使用案例，那么对模型进行训练后分析和训练后量化对于改善模型极为重要。将项目部署到更广泛的受众的效率，质量和紧凑性。 训练后量化还使我们能够在量化模型上实现几乎与原始模型中获得的精度相同的精度。 这简化并简化了我们的生活！

Thank you all for sticking on this the end. Have a great time in the exploration and quantization of your machine learning and deep learning models. Feel free to check out all the previously mentioned projects by me and I hope all of you enjoyed the article and wish you all a wonderful day!

谢谢大家坚持到底。 在探索和量化机器学习和深度学习模型方面花费大量时间。 随时检查我之前提到的所有项目，希望大家都喜欢这篇文章，并祝您生活愉快！

翻译自: https://towardsdatascience.com/post-training-analysis-and-quantization-of-machine-learning-and-deep-learning-models-a133231f6100

机器学习 量化 滚动训练