DeepFake Learning

Awesome-DeepFake-Learning

The approach I work on DeepFake.

• Awesome-DeepFake-Learning
  • 1. Intuitive Learning
  • 2. Survey Paper
  • 3. Curated lists
  • 4. Deepfakes Datasets
  • 5. Generation of synthetic content
    • 5.1 Generation Text
      • ⚒️ Tools ⚒️
      • Papers
      • Webs
      • Awesome
    • 5.2 Generation Audio
      • ⚒️ Tools ⚒️
      • Papers
    • 5.3 Generation Images
      • ⚒️ Tools ⚒️
      • Papers
      • Webs
      • Awesome
    • 5.4 Generation Videos
      • ⚒️ Tools ⚒️
      • Papers
      • Webs
      • Videos
  • 6. Detection of synthetic content
    • 6.1 Detection Text
      • ⚒️ Tools ⚒️
      • Papers
    • 6.2 Detection Audio
      • ⚒️ Tools ⚒️
      • Papers
    • 6.3 Detection Images
      • ⚒️ Tools ⚒️
      • Papers
    • 6.4 Detection Videos
      • ⚒️ Tools ⚒️
      • Papers
      • Videos
      • Awesome
  • 7. Misc
    • Articles
    • Talks
    • Challenges
    • Forums

1. Intuitive Learning

• News From CNN
• 10 DeepFake Examples
• An Introduction to DeepFakes
• 荷兰初创公司Deeptrace发布两个年度（2018年和2019年）Deepfakes发展状况调研报告

2. Survey Paper

• The Creation and Detection of Deepfakes: A Survey
• Media Forensics and DeepFakes: an overview
• DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection
• Deep Learning for Deepfakes Creation and Detection: A Survey

3. Curated lists

• Deep-Learning-for-Tracking-and-Detection: Collection of papers, datasets, code and other resources for object tracking and detection using deep learning

4. Deepfakes Datasets

Datasets	Year	Ratio tampered:original	Total videos	Source	Participants Consent	Tools
UADFV	2018	1 : 1.00	98	YouTube	N	FakeAPP
FaceForensics	2018	1 : 1.00	2008	YouTube	N	Face2Face
Deepfake-TIMIT	2019	1 : 1.00	620	Vid-TIMIT	N	faceswap-GAN
FaceForensics++	2019	1 : 0.25	5000	YouTube	N	faceswap DeepFake Face2Face NeuralTextures
DeepFakeDetection (part of FaceForensics++)	2019	1 : 0.12	3363	Actors	Y
Celeb-DF	2019	1 : 0.51	1203	YouTube	N	a refined version of the DeepFake
DFDC Preview Dataset	2019	1 : 0.28	5214	Actors	Y	Unkonwn

5. Generation of synthetic content

5.1 Generation Text

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
Grover	Grover is a model for Neural Fake News – both generation and detection. However, it probably can also be used for other generation tasks.	https://grover.allenai.org/
gpt-2xy	GPT-2 User Interface based on HuggingFace’s Pytorch Implementation	https://gpt2.ai-demo.xyz/
CTRL	Conditional Transformer Language Model for Controllable Generation	N/A
Talk to Transformer	See how a modern neural network completes your text. Type a custom snippet or try one of the examples	https://talktotransformer.com	N/A
LEO	First intelligent system for creating news in Spanish	N/A	N/A
Big Bird	Bird Bird uses State of the Art (SOTA) Natural Language Processing to aid your fact-checked and substantive content.	BigBirdDemo	N/A
aitextgen	A robust Python tool for text-based AI training and generation using GPT-2.	Demo
GPT-3	GPT-3: Language Models are Few-Shot Learners	N/A

Papers

• Language Models are Unsupervised Multitask Learners
• Saliency Maps Generation for Automatic Text Summarization
• Automatic Conditional Generation of Personalized Social Media Short Texts
• Neural Text Generation in Stories Using Entity Representations as Context
• DeepTingle
• GPT-3: Language Models are Few-Shot Learners
• Evaluation of Text Generation: A Survey

Webs

• NotRealNews
• BotPoet
• TheseLyricsDoNotExist
• ThisResumeDoesNotExist
• NotRealNews
• ThisArtWorkDoesnotExist
• BoredHumans
• GPT-2 Neural Network Poetry
• A.ttent.io
• ThisEpisodeDoesNotExist

Awesome

• awesome-text-generation
• Awesome GPT-3

5.2 Generation Audio

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
Real-Time-Voice-Cloning	Clone a voice in 5 seconds to generate arbitrary speech in real-time	https://www.youtube.com/watch?v=-O_hYhToKoA
Lyrebird	Create your own vocal avatar!	N/A	N/A
Descrypt	Record. Transcribe. Edit. Mix. As easy as typing.	N/A	N/A
Common Voice	Common Voice is Mozilla’s initiative to help teach machines how real people speak.	N/A	N/A
Resemble.ai	Resemble can clone any voice so it sounds like a real human.	N/A	N/A
TacoTron	Tacotron (/täkōˌträn/): An end-to-end speech synthesis system by Google.	Demo
Sonantic	Create a captivating performance using emotionally expressive text-to-speech.	Demo	N/A
15.ai	Natural text-to-speech synthesis with minimal data.	Demo	N/A

Papers

• Neural Voice Cloning with a Few Samples
• Data Efficient Voice Cloning for Neural Singing Synthesis
• Efficient Neural Audio Synthesis
• Score and Lyrics-free Singing Voice Generation
• Generating diverse and natural Text-to-Speech samples using a quantized fine-grained vae and autoregressive prosody prior
• Rave.dj

5.3 Generation Images

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
StyleGAN	An alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis. The new generator improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation.	https://www.youtube.com/watch?v=kSLJriaOumA
StyleGAN2	Improved version for StyleGAN.	https://www.youtube.com/watch?v=c-NJtV9Jvp0
DG-Net	Joint Discriminative and Generative Learning for Person Re-identification	https://www.youtube.com/watch?v=ubCrEAIpQs4
GANSpace	Discovering Interpretable GAN Controls	http://www.exploreganspace.com/
StarGAN v2	StarGAN v2 - Official PyTorch Implementation (CVPR 2020)	https://youtu.be/0EVh5Ki4dIY
Image GPT	Image GPT	N/A
FQ-GAN	Official implementation of FQ-GAN	http://40.71.23.172:8888
EHM_Faces	EHM_Faces is a machine learning project that can generate high-quality, realistic ice hockey player portraits. Primarily meant for the game Eastside Hockey Manager (EHM), this project can generate portraits either one-at-a-time or in batches (the resulting batches are called facepacks).	N/A
Rewriting a Deep Generative Model	Edits the weights of a deep generative network by rewriting associative memory directly, without training data	Demo

Papers

• A Style-Based Generator Architecture for Generative Adversarial Networks
• Analyzing and Improving the Image Quality of StyleGAN
• High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
• Complement Face Forensic Detection and Localization with Facial Landmarks
• Joint Discriminative and Generative Learning for Person Re-identification
• Image2StyleGAN++: How to Edit the Embedded Images?
• StyleGAN2 Distillation for Feed-forward Image Manipulation
• Generative Pretraining from Pixels
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• Feature Quantization Improves GAN Training
• High-Resolution Neural Face Swapping for Visual Effects
• Improving Style-Content Disentanglement in Image-to-Image Translation
• Rewriting a Deep Generative Model

Webs

• ThisPersonDoesNotExist
• WhichFaceIsReal
• ThisRentalDoesNotExist
• ThisCatDoesNotExist
• ThisWaifuDoesNotExist
• thispersondoesnotexist

Awesome

• Awesome Pretrained StyleGAN2

5.4 Generation Videos

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
FaceSwap	Grover is a model for Neural Fake News – both generation and detection. However, it probably can also be used for other generation tasks.	https://www.youtube.com/watch?v=r1jng79a5xc
Face2Face	FaceSwap is a tool that utilizes deep learning to recognize and swap faces in pictures and videos.	N/A
Faceswap	FaceSwap is an app that I have originally created as an exercise for my students in “Mathematics in Multimedia” on the Warsaw University of Technology.	N/A
Faceswap-GAN	Adding Adversarial loss and perceptual loss (VGGface) to deepfakes’(reddit user) auto-encoder architecture.	https://github.com/shaoanlu/faceswap-GAN/blob/master/colab_demo/faceswap-GAN_colab_demo.ipynb
DeepFaceLab	DeepFaceLab is a tool that utilizes machine learning to replace faces in videos.	https://www.youtube.com/watch?v=um7q–QEkg4
Vid2Vid	Pytorch implementation for high-resolution (e.g., 2048x1024) photorealistic video-to-video translation.	https://www.youtube.com/watch?v=5zlcXTCpQqM
DFaker	Pytorch implementation for high-resolution (e.g., 2048x1024) photorealistic video-to-video translation.	N/A
Image Animation	The videos on the left show the driving videos. The first row on the right for each dataset shows the source videos.	https://www.youtube.com/watch?v=mUfJOQKdtAk
Avatarify	Photorealistic avatars for Skype and Zoom. Democratized. Based on First Order Motion Model…	https://www.youtube.com/watch?v=lONuXGNqLO0
Speech driven animation	This library implements the end-to-end facial synthesis model.	N/A

Papers

• HeadOn: Real-time Reenactment of Human Portrait Videos
• Face2Face: Real-time Face Capture and Reenactment of RGB Videos
• Synthesizing Obama: Learning Lip Sync from Audio
• The Creation and Detection of Deepfakes: A Survey

Webs

• DeepFake中文网
• Website for creating deepfake videos with learning
• Deep Fakes Net - Deepfakes Network
• Faceswap is the leading free and Open Source multi-platform Deepfakes software
• Fakening
• DeepFakesWeb

Videos

• How to Animate Image with a Video

6. Detection of synthetic content

6.1 Detection Text

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
Grover	Grover is a model for Neural Fake News – both generation and detection. However, it probably can also be used for other generation tasks.	https://grover.allenai.org/
GLTR	Detecting text that was generated from large language models (e.g. GPT-2).	http://gltr.io/dist/index.html
fake news detection	In this project, we aim to build state-of-the-art deep learning models to detect fake news based on the content of article itself.	Demo
GPTrue or False	Display the likelihood that a sample of text was generated by OpenAI’s GPT-2 model.	N/A	N/A

Papers

• GLTR: Statistical Detection and Visualization of Generated Text
• Human and Automatic Detection of Generated Text
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• The Limitations of Stylometry for Detecting Machine-Generated Fake News

6.2 Detection Audio

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
Spooded speech detection	This work is part of the “DDoS Resilient Emergency Dispatch Center” project at the University of Houston, funded by the Department of Homeland Security (DHS).	N/A
Fake voice detection	This repository provides the code for a fake audio detection model built using Foundations Atlas. It also includes a pre-trained model and inference code, which you can test on any of your own audio files.	N/A
Fake Voice Detector	For “Deep Learning class” at ETHZ. Evaluate how well the fake voice of Barack Obama 1. confuses the voice verification system, 2. can be detected.	N/A
CycleGAN Voice Converter	An implementation of CycleGAN on human speech conversions	https://leimao.github.io/project/Voice-Converter-CycleGAN/

Papers

• Can We Detect Fake Voice Generated by GANs?
• CycleGAN Voice Converter
• The Rise of Synthetic Audio Deepfakes

6.3 Detection Images

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
FALdetector	Detecting Photoshopped Faces by Scripting Photoshop.	https://www.youtube.com/watch?v=TUootD36Xm0

Papers

• Detecting Photoshopped Faces by Scripting Photoshop

6.4 Detection Videos

⚒️ Tools ⚒️

Name	Description	Demo	Popularity
FaceForensics++	FaceForensics++ is a forensics dataset consisting of 1000 original video sequences that have been manipulated with four automated face manipulation methods: Deepfakes, Face2Face, FaceSwap and NeuralTextures.	https://www.youtube.com/watch?v=x2g48Q2I2ZQ
Face Artifacts	Our method is based on the observations that current DeepFake algorithm can only generate images of limited resolutions, which need to be further warped to match the original faces in the source video.	N/A
DeepFake-Detection	Our Pytorch implementation, conducts extensive experiments to demonstrate that the datasets produced by Google and detailed in the FaceForensics++ paper are not sufficient for making neural networks generalize to detect real-life face manipulation techniques.	http://deepfake-detection.dessa.com/projects
Capsule-Forensics-v2	Implementation of the paper: Use of a Capsule Network to Detect Fake Images and Videos.	N/A
ClassNSeg	Implementation of the paper: Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos (BTAS 2019).	N/A
fakeVideoForensics	Fake video detector	https://www.youtube.com/watch?v=8YYRT4lzQgY	[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Ka08jQIx-1599098979903)(https://badgen.net/github/stars/next-security-lab/fakeVideoForensics)]

Papers

• Exposing DeepFake Videos By Detecting Face Warping Artifacts
• DeepFakes: a New Threat to Face Recognition? Assessment and Detection
• FaceForensics++: Learning to Detect Manipulated Facial Images
• Deepfake Video Detection Using Recurrent Neural Networks
• Deep Learning for Deepfakes Creation and Detection: A Survey
• Protecting World Leaders Against Deep Fakes
• Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos
• DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection
• Media Forensics and DeepFakes:
  an overview
• Everybody’s Talkin’: Let Me Talk as You Want
• FSGAN: Subject Agnostic Face Swapping and Reenactment
• Celeb-DF (v2): A New Dataset for DeepFake Forensics
• Deepfake Video Detection through Optical Flow based CNN
• MesoNet: a Compact Facial Video Forgery Detection Network
• Adversarial Deepfakes
• One-Shot GAN Generated Fake Face Detection
• Evading Deepfake-Image Detectors with White- and Black-Box Attacks
• Deepfakes Detection with Automatic Face Weighting
• Unmasking DeepFakes with simple Features
• VideoForensicsHQ: Detecting High-quality Manipulated Face Videos
• Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systems
• Detecting Deepfake Videos: An Analysis of Three Techniques
• OC-FakeDect: Classifying Deepfakes Using One-class Variational Autoencoder

Videos

• Deepfake Detection using LSTM and ResNext CNN
• End-To-End AI Video Generation To Bring Fake Humans To Life
• DeepFake Cyber Security Threats And Opportunities - Matt Lewis

Awesome

• Awesome-Deepfakes-Materials

7. Misc

Articles

• 2020 Guide to Synthetic Media
• Machine Learning Experiments
• Building rules in public: Our approach to synthetic & manipulated media
• Contenido Sintético (parte I): generación y detección de audio y texto
• Contenido Sintético (parte II): generación y detección de imagenes
• Contenido Sintético (parte III): generación y detección de vídeo
• Fake Candidate
• Unraveling the mystery around deepfakes
• Cyber-Security implications of deepfakes
• Deepfake Detection Challenge Results: An open initiative to advance AI
• The Synthetic Media Landscape
• Do (Microtargeted) Deepfakes Have Real Effects on Political Attitudes?

Talks

• ICML 2019 Synthetic Realities
• CCN-CERT: Automatizando la detección de contenido Deep Fake
• TED Talk: Fake videos of real people
• Hacking with Skynet
• RSA: Deep Fakes Are Getting Terrifyingly Real
• CVPR 2020 Workshop on media forensics

Challenges

• NIST: Media Forensics Challenge 2019
• ASVspoof: Automatic Speaker Verification
• Kaggle: DeepFake Detection Challenge
• Fake News Challenge
• Xente: Fraud detection challenge
• Chalearn Multi-modal Cross-ethnicity Face anti-spoofing Recognition Challenge

Forums

• Reddit: MediaSynthesis
• Reddit: Digital Manipulation
• MrDeepFake Forums
• AIVillage

More avaliable: Link