简历造假
GAN and Deepfakes have become more than research topics or engineering toys. Starting as an innovative research concept, now they can be used as a communication weapon. Deepfakes have numerous positive uses cases, but as with every innovation, deepfakes or AI-Generated synthetic media can be used as a weapon to inflict harm on individuals and institutions.
GAN和Deepfake不仅仅是研究主题或工程玩具。 从一个创新的研究概念开始，现在它们可以用作交流武器。 Deepfake有很多积极的用例，但是与每一项创新一样，deepfakes或AI生成的合成媒体都可以用作对个人和机构造成伤害的武器。
The main objective of any countermeasure to mitigating malicious synthetic media’s negative societal impact must be two-fold. One, to reduce the exposure to malicious deepfakes and second, to minimize the damage it can inflict.
缓解恶意合成媒体的负面社会影响的任何对策的主要目标必须是双重的。 一是减少遭受恶意深造的风险，二是将其可能造成的损害降至最低。
To defend the truth and secure freedom of expression, we need a multi-stakeholder and multimodal approach. Collaborative actions and collective techniques across legislative regulations, platform policies, technology countermeasures, and media literacy approaches must provide an effective and ethical response to the threat of malicious deepfakes.
为了捍卫真理和确保言论自由，我们需要采取多方利益相关者和多模式的方法。 跨立法法规，平台政策，技术对策和媒体素养方法的协作行动和集体技术必须对恶意深造的威胁提供有效和合乎道德的回应。
In this article, I will share some of the technical countermeasures to deepfakes.
在本文中，我将分享一些深造产品的技术对策。
I will share my thoughts on Legislative, Platform policies, and Media Literacy countermeasures in a future post.
我将在以后的文章中分享我对立法，平台政策和媒体素养对策的想法。
Since Deepfakes are created using AI, everyone’s first inclination and a simpler assumption are to find a technology solution as a countermeasure to a technical problem. The technical countermeasures are not simple and are immediately evident as the technological development continues to outpace what is possible by AI and GANs.
由于Deepfakes是使用AI创建的，因此每个人的初衷和一个简单的假设就是找到一种技术解决方案来解决技术问题。 随着技术的发展继续超越AI和GAN的能力，技术对策并不简单且立即显而易见。
Technical solutions for deepfakes are categorized into Media Authentication, Provenance, and Detection.
Deepfake的技术解决方案分为“媒体身份验证”，“来源”和“检测”。
认证与来源 (Authentication & Provenance)
Media authentication and provenance tools can verify the authenticity of the content, obtain its origins, and identify its creator. There are multiple industry efforts to build a set of standards and harmonize the techniques to authenticate, prove origins, and identify authoritative sources. It will require a media standards body like Alliance for Open Media (AOM) and broader industry partners to achieve the goal.
媒体认证和出处工具可以验证内容的真实性，获取其来源并确定其创建者。 工业界已经做出了许多努力来建立一套标准并统一技术以进行身份验证，证明来源和确定权威来源。 要实现这一目标，将需要像开放媒体联盟 (AOM)这样的媒体标准机构和更广泛的行业合作伙伴。
媒体认证 (Media authentication)
Authentication includes solutions that verify authenticity across the media’s life cycle using watermarking, media verification markers, chain-of-custody logging, and other tools. Experts suggest that authentication is the most effective way to prevent the spread of fake media.
身份验证包括使用水印，媒体验证标记，产销监管链日志记录和其他工具在整个媒体生命周期中验证真实性的解决方案。 专家建议，身份验证是防止假冒媒体传播的最有效方法。
Digital watermarking technology and media verification markers are commercially available and widely adopted by media organizations and forensics practitioners. This technology may authenticate and track media throughout its life cycle or verify certificates at the endpoint. Media authentication solutions will always have limited reach, given the massive amount of unauthenticated media publicly available on the internet. Still, media authentication can be of value to a broad audience, providing a high assurance of critical pieces of media such as news broadcasts.
数字水印技术和媒体验证标记是可商购的，并被媒体组织和法医从业人员广泛采用。 该技术可以在其整个生命周期中对媒体进行身份验证和跟踪，或者在端点上验证证书。 鉴于Internet上公开提供了大量未经身份验证的媒体，因此媒体身份验证解决方案的范围将始终很有限。 尽管如此，媒体身份验证对于广大受众还是有价值的，它为诸如新闻广播之类的关键媒体提供了高度保证。
Media authentication tools can also help platforms to gather signals to act on the non-authoritative content. The action may include adding a label or even not allowing the content on the platform.
媒体身份验证工具还可以帮助平台收集信号以对非权威内容采取行动。 动作可能包括添加标签，甚至不允许平台上的内容。
媒体来源验证 (Media provenance verification)
Provenance solutions can provide information on media origins, such as a list of major news and other sites the media was published or posted in the past. Reverse image search on the internet is the most effective and the easiest method for detecting a single fake image or thumbnail. Bing or Google reverse image search is an example of a tool that can be used to determine media provenance. If the deepfake was created using another image somewhere on the internet, the original version should appear in that search.
来源解决方案可以提供有关媒体来源的信息，例如主要新闻列表以及该媒体过去发布或发布的其他站点。 互联网上的反向图像搜索是检测单个伪图像或缩略图的最有效，最简单的方法。 Bing或Google反向图像搜索是可用于确定媒体出处的工具的示例。 如果Deepfake是使用Internet上其他位置的图像创建的，则原始版本应出现在该搜索中。
Reverse video search capabilities are not widely available today because of the inherent challenges of searching across video files frame-by-frame.
由于逐帧搜索视频文件所固有的挑战，因此反向视频搜索功能目前尚未广泛使用。
While media provenance solutions are available today, they are limited due to the massive amount of unstructured, inadequately labeled media data available on the internet.
尽管现在可以使用媒体出处解决方案，但是由于互联网上存在大量的非结构化，标签不足的媒体数据，因此它们受到了限制。
YouTube Content ID
YouTube内容ID
YouTube enables the copyright owners to quickly identify and manage their content by a system called Content ID. YouTube will scan any uploaded video against a database of already submitted by content owners. Only copyright owners who meet specific criteria get access to the ContentID. The requirements to get ContentID dictate that owners must own exclusive rights to a substantial body of original material that they upload to the YouTube creator community. There are explicit guidelines on how to use Content ID. YouTube monitor Content ID use and disputes on an ongoing basis to make sure creators follow these guidelines. If a video’s content matches the copyright owners’ work, they can decide what happens to the content, including removal. Erroneous claims to copyright content may also result in termination of the partnership with YouTube and revoking the ContentID.
YouTube使版权所有者可以通过称为Content ID的系统快速识别和管理其内容。 YouTube会根据内容所有者已提交的数据库扫描所有上传的视频。 只有符合特定条件的版权所有者才能访问ContentID。 获取ContentID的要求规定，所有者必须拥有对上传到YouTube创作者社区的大量原始资料的专有权。 有关如何使用Content ID的明确指南。 YouTube会持续监控Content ID的使用和争议，以确保创作者遵守这些准则。 如果视频的内容与版权所有者的作品相匹配，他们可以决定对内容进行什么处理，包括删除。 对版权内容的错误声明也可能导致终止与YouTube的合作关系并撤销ContentID。
Adobe Content Authenticity Initiative
Adobe内容真实性计划
Adobe is creating a system to provide provenance and capture the history for digital media, giving creators a tool to claim authorship and empower consumers to assess whether what they are seeing is trustworthy. The Content Authenticity Initiative (CAI) creates content attribution, which is critical for transparency, understanding, and trust online as inauthentic content proliferates, and powerful editing tools become easier to access. The CAI aims to provide objective facts about how content came to be without judgment. Adobe is focusing on images for initial implementations but intends to specify a uniform method for creating, attaching, and displaying attribution data for any media type.
Adobe正在创建一个系统来提供来源并捕获数字媒体的历史，从而为创作者提供一种声称作者身份的工具，并授权消费者评估他们所看到的内容是否可信。 内容真实性倡议 (CAI)创建内容归因，随着非真实内容的激增，强大的编辑工具变得更易于访问，这对于在线透明，理解和信任至关重要。 CAI旨在提供客观的事实，说明内容如何变得毫无判断。 Adobe致力于最初实现的图像，但打算指定一种统一的方法来创建，附加和显示任何媒体类型的归因数据。
Microsoft AETHER Media Provenance (AMP)
Microsoft AETHER媒体来源(AMP)
Microsoft AMP is a system that ensures the authentication of media via certifying provenance. AMP allows publishers to create one or more signed manifests for a media instance. These manifests are stored in a highly scalable and performant database for a fast lookup from browsers and applications. The manifests can also be registered and signed by chain-of-custody ledgers like blockchain. The default implementation of the chain of custody solution is using Azure Confidential Consortium Framework (CCF). CCF uses both hardware and software to ensure the integrity and transparency of all registered manifests. AMP with CCF will make the media audit easy and enable a consortium of media providers to govern the service. The media authenticity can be communicated to the user via visual elements in the browser, indicating that an AMP manifest has been successfully located and verified.
Microsoft AMP是一个通过证明来源来确保媒体身份验证的系统。 AMP允许发布者为媒体实例创建一个或多个签名清单。 这些清单存储在高度可扩展的高性能数据库中，以便从浏览器和应用程序中快速查找。 清单也可以由诸如区块链之类的监管链分类账进行注册和签名。 产销监管链解决方案的默认实现是使用Azure机密联盟框架 (CCF)。 CCF同时使用硬件和软件来确保所有已注册清单的完整性和透明性。 带有CCF的AMP将使媒体审核变得容易，并使媒体提供商联盟可以管理该服务。 可以通过浏览器中的可视元素将媒体真实性传达给用户，表明已成功找到并验证了AMP清单。
FuJo Provenance
FuJo来源
In November 2018, the institute for future media & journalism announced it would lead a €2.4 million EU project, PROVENANCE, to develop new tools for improving how information is shared and received on social media. PROVENANCE’s mission is to develop an intermediary-free solution for digital content verification, thus giving greater control to social media users. It will also enable and encourage social sharing dynamics in values of trust, openness, and fair participation. The PROVENANCE Verification layer will use advanced tools for multimedia analytics like semantic uplift, image forensics, and cascade analysis to record any modifications to content assets and identify similar content. A personalized Digital Companion will cater to the information needs of end-users. PROVENANCE will use a chain of custody solutions like blockchain to record multimedia content uploaded and registered by content creators or identified for registration by the PROVENANCE Social Network Monitor in a secure and verifiable way manner.
2018年11月，未来媒体与新闻研究所宣布将领导一项耗资240万欧元的欧盟项目PROVENANCE ，以开发新工具来改善社交媒体上信息的共享和接收方式。 PROVENANCE的任务是为数字内容验证开发无中介的解决方案，从而为社交媒体用户提供更大的控制权。 它还将启用和鼓励信任，开放和公平参与的社会共享动力。 来源验证层将使用先进的工具进行多媒体分析，例如语义提升，图像取证和级联分析，以记录对内容资产的任何修改并识别相似的内容。 个性化的数字伴侣将满足最终用户的信息需求。 PROVENANCE将使用诸如区块链之类的监管解决方案链来记录由内容创建者上传和注册的多媒体内容，或以安全且可验证的方式记录由PROVENANCE社交网络监控器识别为注册的多媒体内容。
深度检测 (Deepfake Detection)
Deepfake detection typically includes solutions that leverage multi-modal detection techniques to determine whether target media has been manipulated. So far, most of the detection research and mitigation efforts have focused on automated deepfake detection. Using GANs and other technology, the approaches to generate fake digital content have improved considerably. It presents a cat and mouse game even worse than cybersecurity [1].
Deepfake检测通常包括利用多模式检测技术来确定目标媒体是否已被操纵的解决方案。 到目前为止，大多数检测研究和缓解措施都集中在自动Deepfake检测上。 使用GAN和其他技术，生成伪造数字内容的方法有了很大的改进。 它提出的猫和老鼠游戏甚至比网络安全性还差[ 1 ]。
Automated detection techniques use new and innovative Machine Learning and AI techniques to discern if the media has been manipulated.
自动化检测技术使用创新的机器学习和AI技术来识别媒体是否已被操纵。
基于伪影的检测 (Artifact-Based Detection)
Deepfakes often generate artifacts that are hard for humans to notice. Researchers have proposed a few techniques using machine learning and AI to identify those inconsistencies and detect deepfakes.
Deepfake经常会产生人类难以察觉的伪影。 研究人员提出了一些使用机器学习和AI的技术，以识别那些不一致之处并检测深层造假。
In 2019, Yuezun Li and Siwei Lyu from the University of Albany proposed a solution by detecting face warping artifacts to identify deepfakes [2]. Their detection technique is based on the observations that current deepfake algorithms need warping techniques to match the original faces extracted from the source video frames. The warping techniques will leave distinctive artifacts in the resulting deepfake, which can be adequately captured by convolutional neural networks (CNNs).
在2019年，来自奥尔巴尼大学的李跃尊和吕思玮提出了一种解决方案，该方法通过检测面部翘曲伪像来识别深层假货[ 2 ]。 他们的检测技术基于以下观察：当前的Deepfake算法需要扭曲技术，以匹配从源视频帧提取的原始面部。 扭曲技术将在生成的深层伪造中留下明显的伪像，可以通过卷积神经网络 (CNN)充分捕获。
In a paper FakeCatcher Detection of Synthetic Portrait Videos using Biological Signals, the authors asserted that that biological signals like a heartbeat, pulse, blood volume patterns hidden in portrait videos could be used as an implicit descriptor of authenticity, because they are neither spatially nor temporally preserved in fake content [3].
在一篇使用生物信号进行FakeCatcher检测合成人像视频的论文中，作者断言，人像视频中隐藏的诸如心跳，脉搏，血容量模式之类的生物信号可以用作真实性的隐式描述，因为它们既不在空间上也不在时间上保留为假内容[ 3 ]。
Microsoft Research published a paper to observe the altered face’s blending into the real image as an artifact for detection. [4]. The technology converts the input face image to a greyscale image that can reveal whether the input image can be decomposed into the blending of two images. The algorithm does the detection by identifying the blending boundary for a forged image and the absence of blending for a real image.
Microsoft Research发表了一篇论文，目的是观察改变后的面部是否融合到真实图像中，作为检测伪像。 [ 4 ]。 该技术将输入的面部图像转换为灰度图像，可以显示输入图像是否可以分解为两个图像的混合。 该算法通过识别伪造图像的融合边界和真实图像没有融合的方法进行检测。
基于不一致的检测 (Inconsistency-Based Detection)
Several techniques for identifying inconsistencies in media can be used for deepfake detection. Dubbing inconsistencies between audio speech patterns and mouth motion, speaker features, and visual face features (e.g., a voice change, but no talking face change) inconsistencies can help gain the confidence score for deepfake detection.
可以使用几种用于识别介质不一致的技术来进行Deepfake检测。 音频语音模式与嘴巴动作，说话者特征和视觉面部特征(例如，语音变化，但没有说话的面部变化)之间的配音不一致可以帮助获得用于深度假检测的置信度分数。
In the paper Spotting Audio-Visual Inconsistencies (SAVI) in Manipulated Video, the authors proposed algorithmically finding discrepancies between the type of scenes detected for deepfake detection[5].
在论文《在操纵视频中发现视听不一致》(SAVI)中，作者提出了算法上发现用于深度虚假检测的场景类型之间的差异[ 5 ]。
Video and Image Processing Lab at Purdue University published a paper to use a temporal-aware pipeline to automatically detect deepfake videos[6]. The algorithm leverages the convolutional neural network (CNN) to extract frame-level object features. These features are used to train a recurrent neural network (RNN) that learns to classify by finding temporal inconsistencies if a video has been subject to manipulation or not.
普渡大学的视频和图像处理实验室发表了一篇论文，使用时间感知管道自动检测Deepfake视频[ 6 ]。 该算法利用卷积神经网络( CNN )提取帧级对象特征。 这些功能用于训练一个递归神经网络( RNN )，该视频通过查找时间不一致来学习分类，无论视频是否经过处理。
Generative adversarial networks (GAN) are pushing the limits of image manipulation. GAN leaves its specific fingerprint in the images it generates, just like real-world cameras mark acquired images with traces of their photo-response non-uniformity pattern [7]. The GAN fingerprint and analysis of digital camera unique sensor notice (PNRU) can be used for finding the source of the video and help in detection [8].
生成对抗网络(GAN)推动了图像处理的极限。 GAN将其特定的指纹留在生成的图像中，就像现实世界中的相机用其光响应非均匀性图案的痕迹标记采集的图像一样[ 7 ]。 GAN指纹和数码相机唯一传感器通知(PNRU)分析可用于查找视频源并帮助检测[ 8 ]。
语义检测 (Semantic Detection)
Algorithmic detection techniques that rely on statistical fingerprints and anomalies can be fooled with limited additional resources (algorithm development, data, or compute). Since the existing media generation, deepfakes heavily rely on purely data-driven approaches; they are prone to making semantic errors. DARPA’s SemaFor program is focused on using semantic inconsistencies for detection.
依靠统计指纹和异常的算法检测技术可能会因有限的其他资源(算法开发，数据或计算)而被愚弄。 自从现有媒体产生以来，深造假严重依赖纯粹的数据驱动方法。 他们容易犯语义错误。 DARPA的SemaFor程序专注于使用语义不一致进行检测。
Shruti Agarwal and Hany Farid from Berkeley proposed a soft biometric approach [9]. This forensic technique models facial expressions and movements that typify an individual’s speaking pattern for deepfake detection. Although not visually apparent, these correlations are often violated by the nature of how deep-fake videos are created.
伯克利的Shruti Agarwal和Hany Farid提出了一种软生物识别方法[ 9 ]。 这种取证技术可对表情和动作进行建模，这些表情和动作代表了个人的说话模式，从而可以进行深度检测。 尽管在视觉上不明显，但这些关联常常因创建深度视频的性质而受到侵犯。
其他检测方法 (Other Detection Approaches)
Deep Neural Networks (DNN) can learn the essential features from the media to create a generic classification model. The generalization ability to create images using GANs is needed as more and new approaches to GANs are emerging.
深度神经网络(DNN)可以从媒体学习基本功能，以创建通用分类模型。 随着越来越多的新兴GANs需要使用GANs来创建图像的通用能力。
In the paper, On the generalization of GAN image forensics (Xuan et al., 2019), the authors propose using a preprocessed set of images to train a forensic CNN model [10]. By applying similar image-level preprocessing to both real and fake training images, the forensics model is forced to learn more intrinsic features to classify the generated and authentic face images. In some cases, the DNNs will perform better than traditional digital forensic tools on compresses media.
在关于GAN图像取证的一般化的论文中(Xuan等人，2019)，作者建议使用经过预处理的图像集来训练取证的CNN模型[ 10 ]。 通过对真实和伪造的训练图像都应用相似的图像级别预处理，强制取证模型学习更多的固有特征，以对生成的和真实的面部图像进行分类。 在某些情况下，DNN在压缩媒体上的性能将优于传统的数字取证工具。
In the paper FakeSpotter A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces [11] . The authors propose that monitoring neuron behavior can serve as an asset in detecting fake faces since layer-by-layer neuron activation patterns may capture more subtle features that are important for the fake detector.
在论文FakeSpotter中，一个简单而稳健的基准用于发现AI合成的假人脸[ 11 ]。 作者提出，监视神经元行为可以作为检测假脸的一项资产，因为逐层神经元激活模式可能会捕获更多对伪造检测器重要的细微特征。
结论 (Conclusion)
Since deepfakes are created via adversarial training (mostly GANs), the deepfakes become more believable by attempting to trick an algorithmic detector and iterating on the results. Their ability to evade AI-based detection methods will improve as they are introduced to new detection systems. It provides a “cat-and-mouse” process slanted in favor of the mice, especially in the long term.
由于Deepfake是通过对抗性训练(主要是GAN)创建的，因此通过尝试欺骗算法检测器并迭代结果，Deepfake变得更加可信。 随着它们被引入新的检测系统，它们规避基于AI的检测方法的能力将得到提高。 它提供了有利于小鼠的“猫鼠”过程，特别是从长期来看。
All the deepfake detection countermeasures are focused on solving the problem in the short term, with an expectation that authentication and provenance techniques will be a long-term solution to the deepfake issue area.
所有的Deepfake检测对策都集中在短期内解决问题，并期望认证和出处技术将是Deepfake问题领域的长期解决方案。
翻译自: https://towardsdatascience.com/technical-countermeasures-to-deepfakes-564429a642d3
简历造假