【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations

Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations

CVPR 2023

互联网上丰富的instructional videos及其解说为理解程序性活动提供了令人兴奋的途径。

在这项工作中,作者建议学习视频表征,基于网络instructional videos及其叙述的大规模数据集,在不使用人工注释的情况下,对动作步骤及其时间顺序进行编码。本方法联合学习了一个视频表征来编码单个步骤概念,以及一个深度概率模型来捕获步骤顺序中的时间依赖性和巨大的个体变化。经验证明,学习时间排序不仅可以为过程推理提供新的能力,而且可以加强对单个步骤的识别。本模型在step分类(+2.8%/+3.3%在COIN / EPIC-Kitchens)和step预测(+7.4%在COIN)上显著提高了最新的结果。此外,本模型在step分类和预测的zero-shot推理以及对不完整过程的不同和合理步骤的预测方面取得了很好的结果。

代码:https://github.com/facebookresearch/ProcedureVRL

【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第1张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第2张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第3张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第4张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第5张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第6张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第7张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第8张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第9张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第10张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第11张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第12张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第13张图片
【76】论文阅读Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations_第14张图片
更多论文分享,请参考: 深度学习相关阅读论文汇总(持续更新)

你可能感兴趣的:(深度学习论文阅读,论文阅读)