深度多视图信息瓶颈：Deep Multi-view Information Bottleneck

论文题目：Deep Multi-view Information Bottleneck

Summary

• 单个视图的信息瓶颈模型已经发展较为成熟，但尚未有在多视图上的信息瓶颈理论，本论文将信息瓶颈理论用于处理多视图数据，在学习得到单个视图的表征后，融合生成最终表征。

Problem Statement

• 现有的多视图融合的过程都是线性的，但多个视图之间的联系往往是非线性的而且是复杂的，所以需要更好的融合的过程。

Method

• 信息瓶颈：学习得到标签信息的同时去除冗余信息。
  
• 双视图信息瓶颈：各个视图学习得到去除冗余的表征，并使融合后的表征富含标签信息。
  
• 多视图信息瓶颈：
  

Evaluation

• 合成数据集上进行测试
• 真实数据集上进行测试

Conclusion

• 将信息瓶颈应用于多视图数据处理，增加目标信息，减少冗余信息：The model encouraged the latent representation keeping target information as much as possible while containing the information of original features as little as possible to reduce the model complexity.
• 使用变分估计的方法对互信息进行估计：Since the mutual information terms were intractable, we maximized the lower bound of the formulation instead of directly maximizing it.
• 实验测试结果：We demonstrated experiments on various synthetic and real-world datasets to show the effectiveness of the proposed method.

Notes

• 传统的多视图之间的结合方式大多是线性结合的，但多个视图之间的关系使复杂的，各个视图之间存在共享的信息，也存在互补的信息，因此多视图的表征学习和融合的过程同样重要。
• In many classification problems, the predictions can be enhanced by fusing information from different data views.
• In particular, when the information from different views complement each other, it is expected that
  multi-view learning will lead to improved predictive performance.

References

• Multi-view clustering via canonical correlation analysis.
• Document clustering using word clusters via the information bottleneck method.
• Multimodal learning with deep boltzmann machines.
• A survey of multi-view machine learning.
• On deep multi-view representation learning.
• Large-margin multi-viewinformation bottleneck.