Python开源项目GFPGAN——人脸重建（Face Restoration），模糊清晰、划痕修复及黑白上色的实践

Python Anaconda 的安装、配置等等请参阅：

Python开源项目CodeFormer——人脸重建（Face Restoration），模糊清晰、划痕修复及黑白上色的实践https://blog.csdn.net/beijinghorn/article/details/134334021

TIP: 本项目名义上是 腾讯 LAB，实际上都是一些非中国孩子干的活，模型也都是洋人，所以用来修正中国人图片，效果很一般。速度也很一般。直接用C++代码重写后，速度就快很多了。

5 GFPGAN

GFPGAN aims at developing a Practical Algorithm for Real-world Face Restoration.
It leverages rich and diverse priors encapsulated in a pretrained face GAN (e.g., StyleGAN2) for blind face restoration.
GFPGAN旨在开发一种实用的真实世界的人脸恢复算法。
它利用了封装在预先训练好的面部GAN（如StyleGAN2）中的丰富而多样的先验来进行盲脸修复。
Frequently Asked Questions can be found in FAQ.md.

5.1 概要Summary

5.1.1 进化史Updates

Add RestoreFormer inference codes.
Add V1.4 model, which produces slightly more details and better identity than V1.3.
Add V1.3 model, which produces more natural restoration results, and better results on very low-quality / high-quality inputs. See more in Model zoo, Comparisons.md
Integrated to Huggingface Spaces with Gradio. 
See Gradio Web Demo.Support enhancing non-face regions (background) with Real-ESRGAN.
We provide a clean version of GFPGAN, which does not require CUDA extensions.
We provide an updated model without colorizing faces.

5.1.2 About GFPGAN

If GFPGAN is helpful in your photos/projects, please help to ⭐ this repo or recommend it to your friends. Thanks‘s Other recommended projects:
Real-ESRGAN: A practical algorithm for general image restoration
https://github.com/xinntao/Real-ESRGAN

BasicSR: An open-source image and video restoration toolbox
https://github.com/xinntao/BasicSR

facexlib: A collection that provides useful face-relation functions
https://github.com/xinntao/facexlib

HandyView: A PyQt5-based image viewer that is handy for view and comparison
https://github.com/xinntao/HandyView

5.2 论文Paper

GFP-GAN: Towards Real-World Blind Face Restoration with Generative Facial Prior
[Paper]   [Project Page]   [Demo]
Xintao Wang, Yu Li, Honglun Zhang, Ying Shan
Applied Research Center (ARC), Tencent PCG

5.3 依赖项与安装Dependencies and Installation

5.3.1 依赖项Depandencies

Python >= 3.7 (Recommend to use Anaconda or Miniconda)
PyTorch >= 1.7
Option: NVIDIA GPU + CUDA
Option: Linux

5.3.2 安装Installation

We now provide a clean version of GFPGAN, which does not require customized CUDA extensions.
If you want to use the original model in our paper, please see PaperModel.md for installation.

Clone repo

git clone https://github.com/TencentARC/GFPGAN.git
cd GFPGAN

Install dependent packages

# Install basicsr - https://github.com/xinntao/BasicSR
# We use BasicSR for both training and inference
pip install basicsr

# Install facexlib - https://github.com/xinntao/facexlib
# We use face detection and face restoration helper in the facexlib package
pip install facexlib

pip install -r requirements.txt
python setup.py develop

# If you want to enhance the background (non-face) regions with Real-ESRGAN,
# you also need to install the realesrgan package
pip install realesrgan

5.4 快速指南Quick Inference

We take the v1.3 version for an example. More models can be found here.
https://github.com/TencentARC/GFPGAN#european_castle-model-zoo

Download pre-trained models: GFPGANv1.3.pth
https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth

wget https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth -P experiments/pretrained_models

Inference!

python inference_gfpgan.py -i inputs/whole_imgs -o results -v 1.3 -s 2
Usage: python inference_gfpgan.py -i inputs/whole_imgs -o results -v 1.3 -s 2 [options]...

  -h                   show this help
  -i input             Input image or folder. Default: inputs/whole_imgs
  -o output            Output folder. Default: results
  -v version           GFPGAN model version. Option: 1 | 1.2 | 1.3. Default: 1.3
  -s upscale           The final upsampling scale of the image. Default: 2
  -bg_upsampler        background upsampler. Default: realesrgan
  -bg_tile             Tile size for background sampler, 0 for no tile during testing. Default: 400
  -suffix              Suffix of the restored faces
  -only_center_face    Only restore the center face
  -aligned             Input are aligned faces
  -ext                 Image extension. Options: auto | jpg | png, auto means using the same extension as inputs. Default: auto

If you want to use the original model in our paper, please see PaperModel.md for installation and inference.
https://github.com/TencentARC/GFPGAN/blob/master/PaperModel.md

5.5 模型库Model Zoo

Version    Model Name    Description
V1.3    GFPGANv1.3.pth    Based on V1.2; more natural restoration results; better results on very low-quality / high-quality inputs.
https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth

V1.2    GFPGANCleanv1-NoCE-C2.pth    No colorization; no CUDA extensions are required. Trained with more data with pre-processing.
https://github.com/TencentARC/GFPGAN/releases/download/v0.2.0/GFPGANCleanv1-NoCE-C2.pth

V1    GFPGANv1.pth    The paper model, with colorization.
https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/GFPGANv1.pth

The comparisons are in Comparisons.md.
https://github.com/TencentARC/GFPGAN/blob/master/Comparisons.md

Note that V1.3 is not always better than V1.2. You may need to select different models based on your purpose and inputs.

You can find more models (such as the discriminators) here: [Google Drive], OR [Tencent Cloud 腾讯微云]
https://drive.google.com/drive/folders/17rLiFzcUMoQuhLnptDsKolegHWwJOnHu?usp=sharing
https://share.weiyun.com/ShYoCCoc

5.6 自我训练Training

We provide the training codes for GFPGAN (used in our paper).
You could improve it according to your own needs.

Tips

1.More high quality faces can improve the restoration quality.
2.You may need to perform some pre-processing, such as beauty makeup.

Procedures

(You can try a simple version ( options/train_gfpgan_v1_simple.yml) that does not require face component landmarks.)

1.Dataset preparation: FFHQ
https://github.com/NVlabs/ffhq-dataset

2.Download pre-trained models and other data. Put them in the experiments/pretrained_models folder.
i）Pre-trained StyleGAN2 model: StyleGAN2_512_Cmul1_FFHQ_B12G4_scratch_800k.pth
https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/StyleGAN2_512_Cmul1_FFHQ_B12G4_scratch_800k.pth

ii）Component locations of FFHQ: FFHQ_eye_mouth_landmarks_512.pth
https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/FFHQ_eye_mouth_landmarks_512.pth

iii）A simple ArcFace model: arcface_resnet18.pth
https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/arcface_resnet18.pth

3.Modify the configuration file options/train_gfpgan_v1.yml accordingly.

4.Training

python -m torch.distributed.launch --nproc_per_node=4 --master_port=22021 gfpgan/train.py -opt options/train_gfpgan_v1.yml --launcher pytorch

5.7 权利License and Acknowledgement

GFPGAN is released under Apache License Version 2.0.

BibTeX
@InProceedings{wang2021gfpgan,
    author = {Xintao Wang and Yu Li and Honglun Zhang and Ying Shan},
    title = {Towards Real-World Blind Face Restoration with Generative Facial Prior},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2021}
}

5.8 联系Contact

If you have any question, please email [email protected] or [email protected].