人脸活体检测（一）paper & code

一、 survey

《A Survey on Anti-Spoofing Methods for Facial Recognition with RGB Cameras of Generic Consumer Devices》 2020

https://arxiv.org/pdf/2010.04145.pdf

CNN [70] Face de-spoofing: Anti-spoofing via noise modeling.ECCV 2018.  https://github.com/yaojieliu/EC CV2018-FaceDeSpoofing   134

CNN [71] Deep pixel-wise binary supervision for face presentation attack detection. 2019.     https://github.com/abh799/DeepPixBis    

PESUDO-DEPTH MAP [38]Face anti-spoofing using patch and depth-based cnns. IJCB 2017.  GitHub - shicaiwei123/patch_based_cnn: the implement of Face Anti-Spoofing Using Patch and Depth-Based CNNs

PESUDO-DEPTH MAP [34] Learning deep models for face anti-spoofing: Binary or auxiliary supervision. CVPR 2018.GitHub - huguesva/Face-Anti-Spoofing-Neural-Network: Implementation of "Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision" in PyTorch.77

PESUDO-DEPTH MAP [82] Exploiting temporal and depth information for multi-frame face anti-spoofing.  2018. https://github.com/lauhsu/Face-Anti-Spoofing-Depth

DOMAIN ADAPTION [89] Multi-adversarial discriminative deep domain generalization for face presentation attack detection. CVPR 2019. https://github.com/rshaojimmy/CVPR2019-MADDoG

《Deep Learning for Face Anti-Spoofing: A Survey》2021

https://arxiv.org/pdf/2106.14948.pdf

1、数据集：

1）3DMA : A multi-modality 3d mask face anti-spoofing database

GitHub - xjinchuan/3dma: A Multi-modality 3D Mask Face Anti-spoofing Database  未释放

2）CASIA-SURF: A dataset and benchmark for large-scale multi-modal face anti-spoofing

      CeFA : Casia-surf cefa: A benchmark for multi-modal cross-ethnicity face anti-spoofing

https://github.com/liuajian/Face-Anti-spoofing-Datasets   需申请

3）WMCA :Biometric face presentation attack detection with multi-channel CCN

    https://www.idiap.ch/dataset/wmca    需申请​

     HQ-WMCA: Deep models and shortwave infrared information to detect FPA    

    https://www.idiap.ch/dataset/hq-wmca    需申请

 2、 Evaluation Metrics

1）错误拒绝率(False rejection Rate, FRR)：错误拒绝live accesses的比率

      错误接受率(False acceptance Rate, FAR)：错误接受spoofing attacks的比率

2）总错误率(HTER)：FRR和FAR的平均值

      平均错误率(EER): EER是FAR和FRR相等时的HTER

      曲线下面积(AUC)：表示bonafide和spoofing之间的可分离程度。

3）Attack Presentation Classification Error Rate，APCER:   攻击样本分类错误率

      Bonafide Presentation Classification Error Rate，BPCER:   真实样本分类错误率

      平均分类错误率(Average Classification错误率，ACER):PCER和APCER的均值，评估数据集内性能的可靠性

3、Method

​

a）Binary CE loss：减小类内距离、增大类间距离

[151]Attention-based two-stream convolutional networks for face spoofing detection

GitHub - Vincent9797/Attention-Based-Two-Stream-Convolutional-Networks-for-Face-Spoofing-Detection

[165]On the effectiveness of vision transformers for zero-shot face anti-spoofing

bob / bob.paper.ijcb2021_vision_transformer_pad · GitLab

b）Pixel-wise Supervision

素级监督pixel-wise supervision：提供更细粒度的上下文感知监督信号:

pseudo depth labels:

[38]Face Anti-Spoofing Using Patch and Depth-Based CNNs https://github.com/shicaiwei123/patch_based_cnn

[35] Searching central difference convolutional networks for face anti-spoofing

GitHub - ZitongYu/CDCN: Central Difference Convolutional Networks (CVPR'20)   *******450

binary mask label  

[44]Deep pixel-wise binary supervision for face presentation attack detection

​​​​​​https://github.com/search?q=Deep+pixel-wise+binary+supervision+for+face+presentation+attack+detection

[175] A-deeppixbis: Attentional angular margin for face anti-spoofing

GitHub - gazeai/a-deeppixbis

c) 领域自适应和泛化技术domain adaptation and generalization techniques  实现鲁棒分类。

域适应：利用（无标记）目标领域的知识来弥补源领域和目标领域之间的差距。

[190] Progressive transfer learning for face anti-spoofing. 利用教师网络，通过特征MMD将两个域的成对样本相似度嵌入进行正则化。https://github.com/xiangn95/Face-Anti-Spoofing-With-Deep-Neural-Network-Distillation

[107] Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing   少量目标域数据可进行的无监督/半监督域自适应GitHub - taylover-pei/USDAN-PR: Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

域泛化：假设在可见的多个源域和未见过但相关的目标域下存在一个广义特征空间，在此空间上，从可见的源域中学习的模型可以很好地推广到看不见的目标领域。直接从多个源域学习广义特征表示，而无需访问任何目标数据。

[60] Multi-adversarial discriminative deep domain generalization for face presentation attack detection 对抗判别学习多个源域共享的广义特征空间

GitHub - rshaojimmy/CVPR2019-MADDoG: Pytorch codes for Multi-adversarial Discriminative Deep Domain Generalization for Face Presentation Attack Detection in CVPR 2019

[62] Cross-domain face presentation attack detection via multi-domain disentangled representation learning将广义FAS特征从主体判别特征和域依赖特征中分离出来。抑制相关因素(如相机传感器和照明)，提高对不可见域的泛化能力。

[61] Regularized fine-grained meta face anti-spoofing 在具有丰富领域知识(如辅助深度)的细粒度元学习过程中，通过寻找广义学习方向来正则化模型。

https://github.com/rshaojimmy/AAAI2020-RFMetaFAS

[63] Single-side domain generalization for face anti-spoofing  多源数据集的域泛化https://github.com/taylover-pei/SSDG-CVPR2020 *******

[194] Selfdomain adaptation for face anti-spoofing使用域适配器扩展了基于元学习的域泛化方法，以在推理时利用未标记的目标域数据。

[195]Generalizable representation learning for mixture domain face anti-spoofing 在没有使用域标签的情况下，通过区分域表示迭代划分混合域。

d) 零样本/少样本学习和异常检测框架 对未知人脸PA类型进行检测。

Zero/Few-Shot Learning：零样本学习的目的是从预定义的PA中学习广义和判别特征，用于未知的新型PA检测；少样本学习的目的是通过学习预定义的PA和收集到的很少的新攻击样本，使FAS模型快速适应新的攻击。

[65] Learning meta model for zero-and few-shot face anti-spoofing 

为解决灾难性遗忘，提出了一种连续少样本学习范式，从连续的数据流中逐步扩展获得的知识，并通过元学习解决方案使用少量训练样本检测新的PAs。GitHub - qyxqyx/AIM_FAS: Implementation of the paper "Learning Meta Model for Zero- and Few-shot Face Anti-spoofing"

Anomaly Detection： 假设活样本的特征表示更相似、更紧凑，属于正常类，而欺骗样本的特征由于攻击类型和攻击材料的高方差，在异常样本空间中分布差异较大。基于此假设，异常检测通常首先训练一个可靠的一类分类器，对活样本进行准确的聚类。然后，在活动样本簇边缘之外的任何样本(如未知攻击)都将被检测为攻击。
 

5、多模态深度学习:

飞行时间(Time of Flight, TOF)和三维结构光(3D Structured Light, SL)两种深度传感器已嵌入主流手机平台(如Iphone、三星、OPPO、华为)，为二维欺骗检测提供了精确的人脸三维深度分布。与SL相比，TOF对光照和距离等环境条件更有鲁棒性。

近红外模态是除VIS外的一种互补光谱(900 ~ 1800nm)，它有效地利用了真实人脸和欺骗人脸之间的反射差异，但在远距离成像质量较差。此外，VIS-NIR集成硬件模块对于许多门禁系统具有很高的性价比。

特征级融合: 

[40] Casia-surf: A large-scale multi-modal benchmark for face anti-spoofingSD-Net

使用特征重加权机制，在RGB、深度和近红外模式之间选择信息通道特征。

GitHub - zzzkk2009/casia-surf-2019-codes: code for ChaLearn Face Anti-spoofing Attack Detection Challenge@CVPR2019

116

[222][221]多模态多层融合分支，增强了模式之间的语境线索。

[220]在特征融合前提出了每个模式分支的空间和信道注意模块，增强对个体模式特征的识别。

[218]Facebagnet: Bag-of-local features model for multi-modal face anti-spoofing

采用patch-level输入的CNN欺骗提取特异性的判别特征，并对多模态特征设计了模态特征擦除(Modal Feature Erasing)操作，以防止过拟合和更好的融合。

[99] Casia-surf cefa: A benchmark for multi-modal cross-ethnicity face anti-spoofing

GitHub - AlexanderParkin/CASIA-SURF_CeFA: Face Anti-spoofing Attack Detection Challenge@CVPR2020

128

[231] Cross modal focal loss for rgbd face anti-spoofing

提出了一种跨模态 focal loss来调节各模态的loss贡献，减少过拟合。

bob / bob.paper.cross_modal_focal_loss_cvpr2021 · GitLab    *******

输入级和决策级融合：

[223]通过叠加归一化的图像，从灰度、深度和近红外模式融合合成图像。

[230] Data fusion based two-stage cascade framework for multi-modality face anti-spoofing

构建了一个两阶段级联框架，分别从多重预处理深度输入(即归一化、尺度嵌入和方向嵌入)和复合可见-近红外输入(即堆栈、求和和差分)来表示基于深度和反照率的特征。

GitHub - IeeeXpert/Data-Fusion-based-Two-stage-Cascade-Framework-for-Multi-Modality-Face-Anti-Spoofing: Data Fusion based Two-stage Cascade Framework for Multi-Modality Face Anti-Spoofing

[39] Multimodal face anti-spoofing based on central difference networks

从RGB、深度和近红外模式融合单个模型的预测二进制分数，该方法优于CeFA数据集上的输入和特征级融合。

[219]Feathernets: Convolutional neural networks as light as feather for face anti-spoofing

设计了一种具有集成和级联的决策级融合策略。将几个训练良好的深度模型的评分进行聚合，与IR模型的评分进行级联，最终进行live/spoof分类。

GitHub - SoftwareGift/FeatherNets_Face-Anti-spoofing-Attack-Detection-Challenge-CVPR2019: Code for 3rd Place Solution in Face Anti-spoofing Attack Detection Challenge @ CVPR2019,model only 0.35M!!! 1.88ms(CPU)

837

Cross-Modal Translation跨模态转换:

[229] based on cyclegan 首先提出了一种新颖的多类别（实时/欺骗，真实/合成）image translation cycle-GAN，为RGB人脸图像生成相应的NIR图像，然后从RGB的堆叠输入中学习融合特征，并为FAS生成NIR图像。

[232] based on cyclegan 遵循类似的跨模态转换框架，具有新颖的基于子空间的模态正则化，以从源RGB输入生成高保真目标NIR模态。

在实际部署中存在各种模态组合（例如，RGB-NIR、RGB-D、NIR-D 和 RGB-D-NIR）。为每个多模态组合训练单个模型是非常昂贵和低效的。设计一个动态多模态框架，将学习到的多模态知识传播到各种模态组合中，是实现无限多模态部署的一个可能方向。

6、高效网络体系结构: lightweight, distilled, vision transformer

7、Representation Learning表示学习：

迁移学习transfer learning[141,202]: 利用其他大规模数据集中预先训练的语义特征来缓解过拟合

解纠缠学习disentangled learning[54,104]: 从噪声表示中分离出固有的欺骗线索intrinsic spoofing

8、从各种人脸识别终端连续采集日常未标记的人脸数据，用于半监督学习[191]。

9、如何充分利用未标记的不平衡(即，live>>spoof)数据，避免意外的性能下降。

10、FAS数据增强策略[105]。利用对抗学习在更多样化的领域实现自适应数据增强。

二、other paper

1、Single-Side Domain Generalization for Face Anti-Spoofing. CVPR2020

GitHub - taylover-pei/SSDG-CVPR2020: Single-Side Domain Generalization for Face Anti-Spoofing, CVPR2020

158

2、Face De-Spoofing: Anti-Spoofing via Noise Modeling. ECCV 2018

3、On Disentangling Spoof Trace for Generic Face Anti-Spoofing. ECCV 2020

GitHub - yaojieliu/ECCV20-STDN: Source code for ECCV 2020 paper: On Disentangling Spoof Trace for Generic Face Anti-Spoofing

121

4、Face Anti-Spooing via Disentangled Representation Learning. ECCV 2020

5、Face Anti-Spoofing with Human Material Perception. ECCV 2020

6、Searching Central Difference Convolutional Networks for Face Anti-Spoofing. CVPR2019

https://github.com/ZitongYu/CDCN     450

7、数据增强：

Face Anti-Spoofing: Model Matters, So Does Data.   CVPR2020

8、多模态：

 Cross Modal Focal Loss for RGBD Face Anti-Spoofing. CVPR2021

bob / bob.paper.cross_modal_focal_loss_cvpr2021 · GitLab

9、小样本：

Learning Meta Model for Zero- and Few-shot Face Anti-spoofing

https://github.com/qyxqyx/AIM_FAS

多模态：

837：  GitHub - SoftwareGift/FeatherNets_Face-Anti-spoofing-Attack-Detection-Challenge-CVPR2019: Code for 3rd Place Solution in Face Anti-spoofing Attack Detection Challenge @ CVPR2019,model only 0.35M!!! 1.88ms(CPU)

567:   https://github.com/SeuTao/CVPR19-Face-Anti-spoofing

359 :   GitHub - AlexanderParkin/ChaLearn_liveness_challenge: ChaLearn Face Anti-spoofing Attack Detection Challenge@CVPR2019

128： GitHub - AlexanderParkin/CASIA-SURF_CeFA: Face Anti-spoofing Attack Detection Challenge@CVPR2020

116： GitHub - zzzkk2009/casia-surf-2019-codes: code for ChaLearn Face Anti-spoofing Attack Detection Challenge@CVPR2019

【CASIA-SURF】《A Dataset and Benchmark for Large-scale Multi-modal Face Anti-spoofing》_bryant_meng的博客-CSDN博客_casia-surf dataset