内嵌物理知识神经网络(PINN)文献总结

PINN论文集：包含论文链接以及论文笔记链接

PINN介绍

综述论文

• Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems
• Integrating physics-based modeling with machine learning: A survey

1 定义问题, 建立工程架构

• Integrating physics-based modeling with machine learning: A survey
• Physics-informed neural networks: A deep learning framework for solving forward and inverse
  problems involving nonlinear partial differential equations
  用内嵌物理信息的神经网络求解PDE的源头文章，从数据驱动角度提出PINN，求解PDE正逆问题
• DGM: A deep learning algorithm for solving partial differential equations
  对于高维PDE用数值方法不能求解，提出DGM方法即用神经网络求解高维PDE问题，同时提出用Monte Carlo估计方法代替PDE中的二阶导数能够加速学习
• hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
  采用变分区域的方式对区域内采点，这是一种启发式的踩点方法，在准确性上有所提升.
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
  用PINN求解材料PDE问题中的参数，求解逆问题，使用的最常规的PINN，主要是结合了材料相关背景.
• Physics-Informed Neural Networks for Cardiac Activation Mapping
• DeepXDE: A deep learning library for solving differential equations
  提出了基于PINN的一个求解各类PDE的库.
• Deep learning-based method coupled with small sample learning for solving partial differential equation
  特意将损失项中的data loss提出来说这是小样本学习，相比于Rassi的PINN增加data loss项，并与其对比，复现了文章中的算例，增加了一项loss后在准确度上有所提高

2 网络结构选择

• Prevention is Better than Cure: Handling Basis Collapse and Transparency in Dense Networks
  对于网络设计的思考，提出了Basis Collapse的概念，提出给损失函数增加一个构造的惩罚项，能够使用lower-weight net实现相同甚至更好的函数拟合效果.
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Weak adversarial networks for high-dimensional partial differential equation

3 不确定性结合

• Adversarial Uncertainty Quantification in Physics-Informed Neural Network
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Flow Field Tomography with Uncertainty Quantification using a Bayesian Physics-Informed Neural Network
• Physics-Informed Deep Learning: A Promising Technique for System Reliability Assessment
• A Physics-Data-Driven Bayesian Method for Heat Conduction Problems
• Zhang D, Lu L, Guo L, et al. Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems[J]. Journal of Computational Physics, 2019, 397: 108850.
• Zhu Y, Zabaras N, Koutsourelakis P S, et al. Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data[J]. Journal of Computational Physics, 2019, 394: 56-81.
• Yang L, Meng X, Karniadakis G E. B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data[J]. Journal of Computational Physics, 2021, 425: 109913.
• Gao Y, Ng M K. Wasserstein generative adversarial uncertainty quantification in physics-informed neural networks[J]. Journal of Computational Physics, 2022: 111270.
• Yang L, Zhang D, Karniadakis G E. Physics-informed generative adversarial networks for stochastic differential equations[J]. SIAM Journal on Scientific Computing, 2020, 42(1): A292-A317.
• Molnar J P, Grauer S J. Flow field tomography with uncertainty quantification using a Bayesian physics-informed neural network[J]. Measurement Science and Technology, 2022.
• Daw A, Maruf M, Karpatne A. PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021: 237-247.
• Yang Y, Perdikaris P. Adversarial uncertainty quantification in physics-informed neural networks[J]. Journal of Computational Physics, 2019, 394: 136-152.

4 超参与元学习

• Learning and Meta-Learning of Stochastic Advection-Diffusion-Reaction Systems from Sparse Measuremen
  用PINN求解随机反应扩散方程，这里主要用了元学习中的贝叶斯优化算法对网络层数和宽度进行优化.
• Bilevel Programming for Hyperparameter Optimization and Meta-Learning
• On First-Order Meta-Learning Algorithms

5 区域划分

• DPINN: Distributed physics informed neural network for data-efficient solution to partial differential equations
  考虑到PINN的较大计算域内梯度极度下降时梯度不够稳健，而且PINN的深度随着PDE阶数的增加而增加，由于会导致出现消失梯度，导致学习速率变慢的问题。

• cPINN: Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems
  同样使用区域划分，直接横着划分成矩形区域，而DPINN横着竖着划分为的矩形区域

• Extended Physics-InformedNeural Networks (XPINNs): A Generalized Space-Time Domain

• Decomposition Based Deep Learning Framework
  提出了更灵活分解域的XPINN方法，比cPINN区域分解更灵活，而且使用与所有方程。

• Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems
  对求解区域划分sub-domains，提出conservative physics-informed neural network(cPINN)，这样的好处一个也可以使用深度神经网络在领域，解决方案可能有复杂的结构，而浅层神经网络可以在子领域使用相对简单和平滑的解决方案. 该方法的另一个优点是在优化算法的选择和各种训练参数如残差点、激活函数、网络宽度和深度等方面提供了更多的自由. 本文简要讨论了cPINN中涉及的各种误差形式，如优化误差、泛化误差和近似误差及其来源.

6 逆问题论文阅读

• Neural Network Technique in Some Inverse Problems of Mathematical Physics Vladimir
  损失函数的定义也用到了类似PINN的思路，用了RBF-net，选择Morozov’s condition作为收敛条件（即损失函数中某一项低于界）；增加神经元的方法，看增加了后的损失函数有没有减，有减少说明增加是有益的。
• Deep Neural Network Approach to Forward-Inverse Problems
  主要是对正、逆问题收敛进行了证明，算了几个简单方程
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs II: A class of inverse problems
  对逆问题的泛化误差的界进行了推导，用到的算例方程有真解都可以借鉴。

7 Loss权重修改与优化算法

• DPINN: Distributed physics informed neural network for data-efficient solution to partial differential equations

• A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks

• Neural networks catching up with finite differences in solving partial differential equations in higher dimensions

• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

• Understanding and mitigating gradient pathologies in physics-informed neural networks.
  回顾了科学机器学习的进展，特别是内嵌物理信息的神经网络在预测物理系统输出以及从噪音数据发现潜在的物理方面信息的有效性. 我们还将识别和分析这种方法的基本失效模式，它与在模型训练过程中导致不平衡的反向传播梯度的数值刚度(stiffness)有关. 为了解决这一局限性，我们提出了一种学习率退火算法，该算法在模型训练期间利用梯度统计量来平衡损失函数中不同项之间的相互影响. 我们还提出了一种新的神经网络结构，它对这种梯度病理(gradient pathologies)更有弹性

• Modified physics-informed neural network method based on the conservation law constraint and its prediction of optical solitons
  arxiv, 提出了基于守恒定律的PINN，主要是针对高阶非线性薛定谔方程，结合这个方程特性，有能量守恒方程和动量守恒方程，将其作为约束加入损失函数中，相比于经典PINN（PDE loss， BC loss，IC loss），能够求得更准。

• Self-adaptive physics-informed neural networks using a soft attention mechanism

• Self-adaptive loss balanced physics-informed neural networks for the incompressible navier-stokes equations
  自适应损失函数权重用于流体方程求解

• Understanding and mitigating gradient pathologies in physics-informwd neural networks

• Multi-objective loss balancing for physics-informed deep learning里面对损失函数自适应权重有综述

8 与迁移学习结合

• Transfer learning based multi-fidelity physics informed deep neural network
  利用多置信度求解PDE，先利用low-fidelity data 训练网络模型，然后再利用high-fidelity对网络最后两层微调.
• Transfer learning enhanced physics informed neural network for phase-field modeling of fracture
  对传统的基于残差的PINN，改变了优化对象通过minimize the variational energy of the system. 与传统的基于残差的PINN相比，该方法有两个主要优点。首先，边界条件的施加相对简单，也更稳健. 其次，变分能量函数形式下的导数的阶数比传统PINN中使用的残差形式下的导数的阶数低，因此网络的训练速度更快

9 与元学习结合

• A novel meta-learning initialization method for physics-informed neural networks
  提出了一种用于PINN加速的元初始化方法，从相关任务中学习能够实现在新任务上快速收敛的结果
• Meta-learning PINN loss functions将元学习的思想用于PINN的损失函数中

10 应用论文

10.1 波场

• Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks
  PINN是监督对现实超声表面声波数据的监督学习采集频率为5 MH. 。超声波表面波数据表示为金属板顶表面的表面变形，用激光振动法测量. 利用声波波动方程的物理信息，利用自适应激活函数加速了PINN的收敛. 自适应激活函数在激活函数中使用了一个可伸缩的超参数，该超参数经过优化，可以在网络拓扑结构发生动态变化时获得最佳性能. 主要是利用PINN求解PDE，然后利用自适应激活函数加速（对比之下加速效果也并不明显）
• A physics-informed variational DeepONet for predicting the crack path in brittle materials
  arxiv
• A modified physics-informed neural network with positional encoding
  求解Helmholtz equation，应用领域波场
• WAVEFIELD RECONSTRUCTION INVERSION VIA PHYSICS-INFORMED NEURAL NETWORKS、
  求解Helmholtz equation，应用领域波场

10.2 压力

• Physics-informed neural networks for estimating stress transfer mechanics in single lap joints
  数据驱动求解table的压力和the interphase of a single lap joint

10.3 electromagnetic simulation

• Physics-augmented deep learning for high-speed electromagnetic simulation and
  optimization
  Nature Portfolio Journal preprint

10.4 流体

• FlowDNN: a physics-informed deep neural network for fast and accurate flow prediction
• Physics-informed neural networks for high-speed flows
  隐藏流体力学
• Physics-Informed Neural Networks for Cardiac Activation Mapping
  心脏激活图
• DiscretizationNet: A Machine-Learning based solver for Navier-Stokes Equations using Finite Volume Discretization
• NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations
• Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations

10.5 单独涉及求解方程

• Solving Huxley equation using an improved PINN method
• Physics-informed neural networks for solving nonlinear diffusivity and Biot’s equations
  研究如何扩展物理信息神经网络的方法来解决与非线性扩散率和Biot方程有关的正问题和反问题. 我们探讨了具有不同训练样例大小和超参数选择的基于物理的神经网络的准确性. 研究了随机变量对不同训练实现的影响.
• fPINNs: Fractional Physics-Informed Neural Networks
• A Physics Informed Neural Network Approach to Solution and Identification of Biharmonic Equations of Elasticity
  arxiv
• Deep neural network methods for solving forward and inverse problems of time fractional diffusion equations with conformable derivative
  arxiv, 首次提出用pinn来研究conformable time fractional diffusion
• Learn bifurcations of nonlinear parametric systems via equation-driven neural networks
  arxiv，提出了一种新的机器学习方法，通过所谓的方程驱动神经网络(EDNNs)来计算分岔。该网络由两步优化组成:第一步是通过训练经验解数据来逼近参数的解函数；第二步是利用第一步得到的近似神经网络计算bifurcations(分岔)
• Variational Onsager Neural Networks (VONNs): A thermodynamics-based variational learning strategy for non-equilibrium PDEs
  arxiv，基于Onsager’s variational principle
• DiscretizationNet: A Machine-Learning based solver for Navier-Stokes Equations using Finite Volume Discretization
• NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations
• Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations

10.6 医学

• Physics-informed machine learning improves detection of head impacts
• EP-PINNs: Cardiac Electrophysiology Characterisation using Physics-Informed Neural Networks
  arxiv

10.7 地质勘测

• Physics-informed semantic inpainting: Application to geostatistical modeling

10.8 Subsurface transport

• Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport
  主要利用PINN解决steady-state advection–dispersion problem

10.9 Fiber Optics和材料

• Physics-informed Neural Network for Nonlinear Dynamics in Fiber Optics
  主要就是解决非线性薛定谔方程，定义了Nonlinear Dynamics in Fiber Optics(由薛定谔方程控制)
• Physics-informed neural network for modelling the thermochemical curing process of composite-tool systems during manufactureCMAME

10.10 求解热问题

• Physics-Informed Neural Networks (PINNs) for Heat Transfer Problems
  论文
• A physics-informed deep learning method for solving direct and inverse heat conduction problems of materials
• Heat Transfer Prediction With Unknown Thermal Boundary Conditions Using Physics-Informed Neural Networks
• A physics-informed machine learning approach for solving heat transfer equation in advanced manufacturing and engineering applications
• Data-driven modeling for boiling heat transfer: Using deep neural networks and high-fidelity simulation results

10.11 内嵌物理神经网络的重构问题

• Review of piezoelectric impedance based structural health monitoring: Physics-based and data-driven methods，Advances in Structural Engineering
• Reconstructing a dynamical system and forecasting time series by self-consistent deep learning
• Image-based reconstruction for a 3D-PFHS heat transfer problem by ReConNN

11 物理知识与数值方法结合

• Physics-informed dynamic mode decomposition
  将物理知识嵌入动态模态求解中提出了physics-informed DMD (piDMD) optimization

12 PINN的加速研究

13 基于PINN的求解库

• DEEPXDE: A DEEP LEARNING LIBRARY FOR SOL VING DIFFERENTIAL EQUATIONS

• IDRLnet: A Physics-Informed Neural Network Library

• PND: Physics-informed neural-network software for molecular dynamics application
  Integrating physics-based modeling with machine learning: A survey
  PINN作为一种发展了近五年的方法，算法本身也非常容易实现。因此基于各种语言或者框架，PINN也有了若干个求解库了。

• DEEPXDE: A DEEP LEARNING LIBRARY FOR SOL VING DIFFERENTIAL EQUATIONS
  DeepXDE，布朗大学Lu博士开发的，就是DeepONet那位Lu博士。他们组是本次PINN潮流的先驱，应该算是第一款也是“官方”的PINN求解器。集成了基于残差的自适应细化（RAR），这是一种在训练阶段优化残差点分布的策略，即在偏微分方程残差较大的位置添加更多点。还支持基于构造实体几何 （CSG） 技术的复杂几何区域定义。

• NeuroDiffEq，基于PyTorch。NeuroDiffEq通过硬约束来构造NN满足初始/边界条件，细分下来叫PCNN（Physics Constrained Neural Network），由于要设计特定的边界，这种方式会受限于对边界的具体形式。

• Modulus，Nvidia公司发布的，之前叫做SimNet，既然是显卡公司开发的，那么或许可以期待有比较好的硬件性能优化大型工业算例。

• SciANN，基于Keras包封装的实现的。SciANN里面有比较丰富的应用示例，包括弹性、结构力学和振动应用等。基于这个库有了不少文章。

• NeuralPDE.jl，看名字就知道是基于Julia语言开发的，是SciML大项目的一部分。
  ADCME，基于TensorFlow开发的，有一些非线性方程的例子，比如非线性弹性、NS问题和Burgers方程。

• TensorDiffEq，看名字就知道是基于Tensorflow，特点是做分布式计算。主旨是通过可伸缩（scalable）计算框架来求解PINN，明显是为大规模工业应用做铺垫。

• IDRLnet，国内团队发布的基于Pytorch和sympy的开源求解器，包含了鲁棒参数估计、变分极小化问题（比如极小曲面计算）、积分方程求解、参数化代理模型等基础算例。

• Elvet，可以求解PDE和变分极小化问题（如悬链线计算）的Python库。

• Nangs，Python框架，貌似没有更新了。

• PyDEns，一个小型框架，貌似没有更新了

14 多保真与PINN结合

• A physics-informed multi-fidelity approach for the estimation of differential equations parameters in low-data or large-noise regimes
• Multi-fidelity sensor selection: Greedy algorithms to place cheap and expensive sensors with cost constraints
  PySensors: A Python Package for Sparse Sensor Placement
• L. Partin, G. Geraci, A. Rushdi, M. S. Eldred, and D. E. Schiavazzi, ‘MULTIFIDELITY DATA FUSION IN CONVOLUTIONAL ENCODER/DECODER NETWORKS’, p. 24, 2022.
• M. Boodaghidizaji, M. Khan, and A. M. Ardekani, ‘Multi-fidelity modeling to predict the rheological properties of a suspension of fibers using neural networks and Gaussian processes’, Physics of Fluids, vol. 34, no. 5, p. 053101, May 2022, doi: 10.1063/5.0087449.
• N. Black and A. R. Najafi, ‘Learning finite element convergence with the Multi-fidelity Graph Neural Network’, Computer Methods in Applied Mechanics and Engineering, vol. 397, p. 115120, Jul. 2022, doi: 10.1016/j.cma.2022.115120.
• M. Aliakbari, M. Mahmoudi, P. Vadasz, and A. Arzani, ‘Predicting high-fidelity multiphysics data from low-fidelity fluid flow and transport solvers using physics-informed neural networks’, International Journal of Heat and Fluid Flow, vol. 96, p. 109002, Aug. 2022, doi: 10.1016/j.ijheatfluidflow.2022.109002.
• F. Regazzoni, S. Pagani, A. Cosenza, A. Lombardi, and A. Quarteroni, ‘A physics-informed multi-fidelity approach for the estimation of differential equations parameters in low-data or large-noise regimes’, Atti Accad. Naz. Lincei Cl. Sci. Fis. Mat. Natur., vol. 32, no. 3, pp. 437-470, Dec. 2021, doi: 10.4171/RLM/943.
• S. Pawar, O. San, P. Vedula, A. Rasheed, and T. Kvamsdal, ‘Multi-fidelity information fusion with concatenated neural networks’. 2021.
• R. Leiteritz, P. Buchfink, B. Haasdonk, and D. Pflüger, ‘Surrogate-data-enriched Physics-Aware Neural Networks’, arXiv:2112.05489 [cs], Dec. 2021, Accessed: Dec. 17, 2021. [Online]. Available: http://arxiv.org/abs/2112.05489
• X. Meng and G. E. Karniadakis, ‘A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems’, Journal of Computational Physics, vol. 401, p. 109020, Jan. 2020, doi: 10.1016/j.jcp.2019.109020.

15 其他

• Discovering Physical Concepts with Neural Networks

原创不易，转载请带上原文链接，喜欢可关注，后续会不断更新

https://zhuanlan.zhihu.com/p/468748367