PyTorch模型部署建议方案

在这个存储库中，我将分享一些关于在生产环境中部署基于深度学习模型有用的注释和参考资料。

Convert PyTorch Models in Production:

• PyTorch Production Level Tutorials [Fantastic]
• The road to 1.0: production ready PyTorch
• PyTorch 1.0 tracing JIT and LibTorch C++ API to integrate PyTorch into NodeJS [Good Article]
• Model Serving in PyTorch
• PyTorch Summer Hackathon [Very Important]
• Deploying PyTorch and Building a REST API using Flask [Important]
• PyTorch model recognizing hotdogs and not-hotdogs deployed on flask
• Serving PyTorch 1.0 Models as a Web Server in C++ [Useful Example]
• PyTorch Internals [Interesting & Useful Article]
• Flask application to support pytorch model prediction
• Serving PyTorch Model on Flask Thread-Safety
• Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX
• Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX (Another Version)
• EuclidesDB - multi-model machine learning feature database with PyTorch
• EuclidesDB - GitHub
• WebDNN: Fastest DNN Execution Framework on Web Browser
• FastAI PyTorch Serverless API (with AWS Lambda)
• FastAI PyTorch in Production (discussion)

Convert PyTorch Models to C++:

• Loading a PyTorch Model in C++ [Fantastic]
• PyTorch C++ API [Bravo]
• An Introduction To Torch (Pytorch) C++ Front-End [Very Good]
• Blogs on using PyTorch C++ API [Good]
• ATen: A TENsor library
• Important Issue about PyTorch-like C++ interface
• PyTorch C++ API Test
• PyTorch via C++ [Useful Notes]
• AUTOGRADPP
• PyTorch C++ Library
• Direct C++ Interface to PyTorch
• A Python module for compiling PyTorch graphs to C

Deploy TensorFlow Models in Production:

• How to deploy Machine Learning models with TensorFlow - Part1
• How to deploy Machine Learning models with TensorFlow - Part2
• How to deploy Machine Learning models with TensorFlow - Part3
• Neural Structured Learning (NSL) in TensorFlow [Great]
• Building Robust Production-Ready Deep Learning Vision Models
• Creating REST API for TensorFlow models
• “How to Deploy a Tensorflow Model in Production” by Siraj Raval on YouTube
• Code for the “How to Deploy a Tensorflow Model in Production” by Siraj Raval on YouTube
• How to deploy an Object Detection Model with TensorFlow serving [Very Good Tutorial]
• Freeze Tensorflow models and serve on web [Very Good Tutorial]
• How to deploy TensorFlow models to production using TF Serving [Good]
• How Zendesk Serves TensorFlow Models in Production
• TensorFlow Serving Example Projects
• Serving Models in Production with TensorFlow Serving [TensorFlow Dev Summit 2017 Video]
• Building TensorFlow as a Standalone Project
• TensorFlow C++ API Example
• TensorFlow.js
• Introducing TensorFlow.js: Machine Learning in Javascript

Convert Keras Models in Production:

• Deep learning in production with Keras, Redis, Flask, and Apache [Rank: 1st & General Usefult Tutorial]
• Deploying a Keras Deep Learning Model as a Web Application in Python [Very Good]
• Deploying a Python Web App on AWS [Very Good]
• Deploying Deep Learning Models Part 1: Preparing the Model
• Deploying your Keras model
• Deploying your Keras model using Keras.JS
• “How to Deploy a Keras Model to Production” by Siraj Raval on Youtube
• Deploy Keras Model with Flask as Web App in 10 Minutes [Good Repository]
• Deploying Keras Deep Learning Models with Flask
• keras2cpp

Deploy MXNet Models in Production:

• Model Server for Apache MXNet
• Running the Model Server
• Exporting Models for Use with MMS
• Single Shot Multi Object Detection Inference Service
• Amazon SageMaker
• How can we serve MXNet models built with gluon api
• MXNet C++ Package
• MXNet C++ Package Examples
• MXNet Image Classification Example of C++
• MXNet C++ Tutorial
• An introduction to the MXNet API [Very Good Tutorial for Learning MXNet]
• GluonCV
• GluonNLP
• Model Quantization for Production-Level Neural Network Inference [Excellent]

Deploy Machine Learning Models with Go:

• Cortex: Deploy machine learning models in production
• Cortex - Main Page
• Why we deploy machine learning models with Go — not Python

Huawei Deep Learning Framework:

• MindSpore - Huawei Deep Learning Framework
• MindSpore - Tutorial

General Deep Learning Compiler Stack:

• TVM Stack

Model Conversion between Deep Learning Frameworks:

• ONNX (Open Neural Network Exchange)
• Tutorials for using ONNX
• MMdnn [Fantastic]
• Convert Full ImageNet Pre-trained Model from MXNet to PyTorch [Fantastic, & Full ImageNet model means the model trained on ~ 14M images]

Some Caffe2 Tutorials:

• Mnist using caffe2
• Caffe2 C++ Tutorials and Examples
• Make Transfer Learning of SqueezeNet on Caffe2
• Build Basic program by using Caffe2 framework in C++

Some Useful Resources for Designing UI (Front-End Development):

• ReactJS vs Angular5 vs Vue.js
• A comparison between Angular and React and their core languages
• A Guide to Becoming a Full-Stack Developer [Very Good Tutorial]
• Roadmap to becoming a web developer in 2018 [Very Good Repository]
• Modern Frontend Developer in 2018
• Roadmap to becoming a React developer in 2018
• 2019 UI and UX Design Trends [Good]
• Streamlit [The fastest way to build custom ML tools]
• Web Developer Monthly
• 23 Best React UI Component Frameworks
• 9 React Styled-Components UI Libraries for 2018
• 35 New Tools for UI Design
• 5 Tools To Speed Up Your App Development [Very Good]
• How to use ReactJS with Webpack 4, Babel 7, and Material Design
• Adobe Typekit [Great fonts, where you need them]
• Build A Real World Beautiful Web APP with Angular 6
• You Don’t Know JS
• JavaScript Top 10 Articles
• Web Design with Adobe XD
• INSPINIA Bootstrap Web Theme
• A Learning Tracker for Front-End Developers
• The best front-end hacking cheatsheets — all in one place [Useful & Interesting]
• GUI-fying the Machine Learning Workflow (Machine Flow)
• Electron - Build cross platform desktop apps with JavaScript [Very Good]

Mobile & Embedded Devices Development:

• PyTorch Mobile [Excellent]
• Mobile UI Design Trends In 2018
• ncnn - high-performance neural network inference framework optimized for the mobile platform [Useful]
• Alibaba - MNN
• Awesome Mobile Machine Learning
• EMDL - Embedded and Mobile Deep Learning
• Fritz - machine learning platform for iOS and Android
• TensorFlow Lite
• NVIDIA Jetson Inference [Great]

Back-End Development Part:

• Modern Backend Developer in 2018
• Deploying frontend applications — the fun way [Very Good]
• RabbitMQ [Message Broker Software]
• Celery [Distributed Task Queue]
• Kafka [Distributed Streaming Platform]
• Kubernetes - GitHub
• Jenkins and Kubernetes with Docker Desktop
• Create Cluster using docker swarm
• deepo - Docker Image for all DL Framewors
• Kubeflow [deployments of ML workflows on Kubernetes]
• MLOps References [DevOps for ML]
• Data Version Control - DVC [Great]
• PySyft - A library for encrypted, privacy preserving deep learning

GPU Management Libraries:

• GPUtil
• py3nvml [Python 3 binding to the NVIDIA Management Library]
• PyCUDA - GitHub
• PyCUDA
• PyCUDA Tutorial
• setGPU
• Monitor your GPUs [Excellent]
• Grafana - Monitoring and Observability [Excellent]
• Prometheus [Excellent for monitoring solution & extract required metrics]

Speed-up & Scalabale Python Codes:

• Numba - makes Python code fast
• Dask - natively scales Python
• What is Dask
• Neural Network Distiller [Distillation & Quantization of Deep Learning Models in PyTorch]
• PyTorch Pruning Tutorial
• PocketFlow - An Automatic Model Compression (AutoMC) framework [Great]
• Introducing the Model Optimization Toolkit for TensorFlow
• TensorFlow Model Optimization Toolkit — Post-Training Integer Quantization
• TensorFlow Post-training Quantization
• Dynamic Quantization in PyTorch
• Static Quantization in PyTorch
• NVIDIA DALI - highly optimized data pre-processing in deep learning
• Horovod - Distributed training framework
• ONNX Float32 to Float16
• Speeding Up Deep Learning Inference Using TensorRT
• JAX - Composable transformations of Python+NumPy programs

Other:

• A Guide to Production Level Deep Learning
• Facebook Says Developers Will Love PyTorch 1.0
• Some PyTorch Workflow Changes
• wandb - A tool for visualizing and tracking your machine learning experiments
• PyTorch and Caffe2 repos getting closer together
• PyTorch or TensorFlow?
• Choosing a Deep Learning Framework in 2018: Tensorflow or Pytorch?
• Deep Learning War between PyTorch & TensorFlow
• Embedding Machine Learning Models to Web Apps (Part-1)
• Deploying deep learning models: Part 1 an overview
• Machine Learning in Production
• how you can get a 2–6x speed-up on your data pre-processing with Python
• Making your C library callable from Python
• MIL WebDNN
• Multi-GPU Framework Comparisons [Great]