文献阅读（1）--TNNLS 论文：一种用于轴承故障诊断的变分transformer

1. 摘要

Deep learning technology provides a promising approach for rotary machine fault diagnosis (RMFD), where vibration signals are commonly utilized as input of a deep network model to reveal the internal state of machinery. However, most existing methods fail to mine association relationships within signals. Unlike deep neural networks, transformer networks are capable of capturing association relationships through the global self-attention mechanism to enhance feature representations from vibration signals. Despite this, transformer networks cannot explicitly establish the causal association between signal patterns and fault types, resulting in poor interpretability. To tackle these problems, an interpretable deep learning model named the variational attention-based transformer network (VATN) is proposed for RMFD. VATN is improved from transformer encoder to mine the association relationships within signals. To embed the prior knowledge of the fault type, which can be recognized based on several key features of vibration signals, a sparse constraint is designed for attention weights. Variational inference is employed to force attention weights to samples from Dirichlet distributions, and Laplace approximation is applied to realize reparameterization. Finally, two experimental studies conducted on bevel gear and bearing datasets demonstrate the effectiveness of VATN to other comparison methods, and the heat map of attention weights illustrates the causal association between fault types and signal patterns.

深度学习技术提供了一个具有前景的回转机械故障诊断方法(RMFD), 振动信号通常被利用作为深度学习网络模型的输入, 其能够用于揭示了机器的内部状态。然而,大多数现有方法无法信号的关系。与深层神经网络不同,变压器网络6能够捕捉协会关系通过全局自注意力机制来提高从振动信号的特征表示。尽管如此,transformer网络不能明确信号模式之间建立因果联系和故障类型,导致较差的可解释性。为应对这些问题,一个可解释的深度学习模型提出来了，命名为变分注意力的transformer网络（VATN），用于（ RMFD）任务。VATN改进自transformer编码器，来挖掘信号之间的关联关系。嵌入故障类型的先验知识,可以识别振动信号的几个关键特性, 设计了稀疏约束权重用于注意力模型的权重。变分个推理采用强迫关注权重样本狄利克雷分布,并应用拉普拉斯近似实现重新参数化。最后,在两个数据集：伞齿轮、轴承数据集上进行实验，研究证明其他比较方法的有效性,和注意力热图重量说明了故障类型和信号之间的因果关联模式。

2. 内容概览

提出的 variational attention 主要有以下几个优点：

• exquisitely capture impulsive segments and pay more attention to them in feature learning, while the attention of dot-product attention is more distracted.
• variational attention is much sparser than dot-product.
• Although dot-product attention can occasionally capture impulsive segments, it also focuses on other stable segments, taking away the model’s resources.
• Thus, the designed sparsity constraint prompts the model focus on more essential segments.

感觉简单说就是能够提取到冲击特征，从图8 图9来看的话，相比于dot-product方式更加合适吧。

为了证明提出方法能够使得attention 权重更加稀疏，图9展示了这些模型的注意力模型的权值直方图以及核密度估计结果。（有待商榷）

3.主要创新点

（1）提出了一个名为VATN的新模型，该模型可以通过全局注意机制有效地提取信号中的关联关系。
（2）变分推理被用于在注意力权重模型上增加一个稀疏约束，来增强对冲击信号的识别，对fault-inrrelevant信号的滤除。
（3）注意力权重的热图被可视化，以在事后分析期间提供信号模式和故障类型之间的因果关联，并为RMFD提出了一种新的带有VATN的故障诊断框架，该框架在实际工业中具有应用前景。

文献

【1】Variational Attention-Based Interpretable Transformer Network for Rotary Machine Fault Diagnosis