The vulnerabilities of Cybersecurity and future of supply chains

Deepfake: What is real? *************************** and Dr. Ayoub

Sample University

Abstract: The rise of deepfake technology using artificial intelligence (AI) to create synthetic media is a growing concern in the information technology (IT) field. The improved tactics, techniques and procedures used to create these synthetic audio and video files are causing a stir among cybersecurity experts because average users and even skilled technicians find it difficult to tell the difference between valid footage and attempts to counterfeit images. We seek to define deepfake technology, explore how the images are created, provide some positive uses, identify some negative results and reveal efforts to detect fraudulent videos in the wild. Keywords: Deepfake, artificial intelligence, machine learning, face swap, synthetic media, lip synch, puppet master

I. Introduction This a warning shot over the bow of the vessel we call the internet from one of the first researchers to analyze 84 publicly available online news articles on the topic of deepfake technology, which is artificial intelligence used to create synthetic media The definition deepfakes comes from the terms “deep learning” and “fake” caused by techniques that can superimpose facial images of a target person onto a source video to make the target person appear to say and do things in the clip. The more concise definition is using artificial intelligence to synthesize content that frequently is called “face swap,” “lip sync” or

“puppet master,” depending on the level of sophistication and intent (Nguyen, 2021). The issue of sophistication is becoming less important because more tools are available for novice users to create convincing videos. At least 15 of these tools are accessible on the internet and most are available on the GitHub software development platform with names such as AvatarMe, DeepFaceLab, DFaker, DiscoFaceGAN and Faceswap. The number of academic peer-reviewed research papers on the topic grew from about 84 in 2018 to 1,268 in 2020, or more than 15 times for an increase of about 1,509.52 percent in just three years. To put things in perspective, one in five people get their news via social media on the internet on YouTube, second only to Facebook (Westerlund, 2019). The concern is that obtaining information masquerading as news gives credibility to socalled face swaps that leave little trace of manipulation in the form of doctored videos. It is necessary to understand deepfakes to fight the potential damage such technology can create in the wrong hands. The use of artificial intelligence to make this happen allows the deepfake tool to create a realistic likeness without human intervention and automates the process. Some of the earliest technology of this kind was during the U.S. Civil War when then President Abraham Lincoln’s face was superimposed onto an 1852 print of John Calhoun. As recently as 2017, an anonymous user with the handle “deepfake” created a pornographic video using deepfake technology that depicted

Gal Gadot, Scarlett Johansson and Taylor Swift. Most of these images were removed from online platforms, but the concern led to a ban on deepfake images on reputable websites. The Defense Advance Research Project Agency (DARPA) of the U.S Department of Defense reported in July 2017, a fake video depicting former President Barrack Obama was released by researchers using this technology and in May 2018, a low-quality deepfake video of former President Donald Trump was released that appeared to encourage Belgians to withdraw from the Paris Climate Change agreement. The structure for the remainder of this paper includes Section II that describes how it works, Section III reveals positive uses, Section IV highlights negative uses, Section V unearths detection methods and Section VI is a conclusion on the future of deepfake technology.

II. How it works Malicious actors previously created so-called “cheap fakes” that were crude and easily identifiable doctored images as parodies or satire. However, software that can be used to create high-quality deepfakes is readily available and user-friendly to those with little to no technical knowledge or artistic expression. The basic concept is simple. The software finds similar elements in the target and source images, such as eyes, nose and mouth, and then makes up the differences between them. However, the amount of data initially required was astronomical in the thousands of images, but detection was no less cumbersome. The creator must feed thousands of video clips depicting the subjects of the process into the software algorithms, so that the artificial

intelligence can “learn” to mimic a person’s facial expressions, voice, tone, inflection and mannerisms.

Figure 1: A deepfake creation model using two encoder-decoder pairs (Nguyen, 2021). There are at least four major types of deepfake producers that include:

(1) Communities of deepfake hobbyists. (2) Political players such as foreign

governments and a variety of activists. (3) Other malicious actors such as criminals. (4) Legitimate actors, such as television companies.

A recently established deepfake online group reached 90,000 members in a very short period. It is nearly impossible to track that many people sharing tactics, techniques and procedures (TTPs) to create synthetic media. Some of the benign hobbyists see this technology as an intellectual puzzle rather than a way to threaten people or damage reputations. Our research reported some deepfake followers were interested in teaching others about the technology and scaring them into action to obtain paid work for technical services.

Still others are focused on using these deepfake TTPs to make the public question all the videos they see. Some deepfake programs use Google’s Image Search function to explore different social media sites for source data and use those images to replace facial data on its own. A positive use of these techniques is to enhance images that lost quality due to compression and make them more appealing. Generative learning models and dimensionality reduction autoencoders are used to create compact representations of images. Autoencoders minimize data loss and can be used to extract a representation of the image. Another deepfake technique is to use sets of encoders-decoders with split loads for the encoder network. Finding a way to force both faces onto the encoder is what makes deepfake possible. This is achieved when the first input face is encoded and then decoded from a decoder from another facial image. This is repeated for every frame throughout the video as needed. Two sets of training images are needed to teach the artificial intelligence portion of the deepfake program – one group of images from the original to be replaced and another batch from the target that needs to be superimposed on the original image. The images can be enhanced with other photos during the process to achieve better results and the process is faster when the lighting in the different images is similar.

Figure 2: The output images are generated by copying a specific subset of styles from Source B and copying the rest from Source A (Nguyen, 2021). The actual deepfake process uses thousands of both original source images and selected target replacement images via a five-step process. Step one – An encoder is used to encrypt each image and a decoder is used to decrypt each image. The nature of these systems is that the encoder only extracts enough information required to reconstruct the original image to save space. The decoder uses multiple instances of back propagation to match the images through multiple graphical processing units (GPUs), using the decoder from the original image to reconstruct the target image to be inserted in the video. Step two – After this training process is accomplished and the machine “learns” what to do, the new person’s face is swapped in the original image frame by frame. Therefore, features of the person in the original video are drawn onto the second person and the images are merged. Step three – Thousands of images were needed before the use of artificial intelligence to train the machine. However, improving the quality of these input photos greatly enhances the

quality of the output base on angle, definition and lighting. Step four – Deepfake facial images look unrealistic if they are different from the original photo. This can be corrected by reducing the image size to 256 X 256. It is matched to 160 X 160 or 64 X 64-pixel-sized training images, and then reconstructed to full size. However, some of those images can appear blurry. Current technology solves this problem by (1) training a neural network that can work with larger pixel-sized images or (2) reduce the resolution of the image to be swapped (Albahar, 2019). Step five – The deepfake tool examines the new video using technical artificial intelligence aspects known as a discriminator and a generator, which work together. The generator creates the videos and the discriminator attempts to determine whether the video is authentic. When the discriminator finds a fake, it creates a “clue” in a set of parameters that it records so the generator remembers how to avoid that to create the next video better. This creates a Generative Adversarial Network (GAN) to tell the system the type of output required. So, the artificial intelligence “learns” by the interaction loop between the generator and the discriminator. Deepfake videos also focus on emotions, facial expressions and lighting by creating a library for the tool to “learn” how to create better videos in the future. Many of these parameters focus on regions in the images rather than on individual pixels to save space. One version is called a histogram of variants (HOG) pattern.

Figure 3: The GAN architecture consists of a generator and a discriminator implemented by a neural network using back propagation (Nguyen, 2021).

III. Positive uses Positive uses for deepfake technology include education and training. The face of one professor could be superimposed upon another giving a lecture which might encourage students to be more attentive or seem to make an instructor a subject matter expert. The movie industry can and does use such technology to change their storyline, enhance transitions and update content for subsequent releases. A video can be modified using the original actor’s image without having to coordinate the reshoot by gathering the cast and crew in a single location. Many older movies are updated with synthetic media to reflect changes in the current culture. While many consider deepfake technology a violation of privacy, it can enhance these privacy concerns at a digital level. If a child or student’s image is captured in a valuable video, it is possible to alter the faces of bystanders to protect that secondary person’s identity and still disseminate the original content. Multimedia applications using deepfake technology can make a character’s mouth

movements and facial expressions match that of an actor in a sound studio, which is beneficial for the gaming industry. Deepfake technology can be used to change the language spoken in original videos to something else and those without speech can generate their own avatars that speak as they would if they could.

IV. Negative uses Advances in technology made manipulation of photos and videos very easy and likely will require greater scrutiny of such evidence used in court. The most famous use of deepfake is the practice of putting facial images of famous people on actors in pornographic movies. This allows the creator to seek revenge or blackmail a victim. The technique uses thousands of digital images and audio files to build a model of a person saying or doing something he or she did not do. Advancements in technology mean greater care will be required when admitting digital evidence in court. Reports depicting so-called fake news clips can be used to sway public opinion against a political or spiritual leader. Negative deepfake reports can disrupt the financial markets by suggesting a particular company or business leader was doing something inappropriate and watch the stocks take a plunge. Blackmail, revenge, propaganda and sabotage are all reasons behind negative uses for deepfake technology.

V. Detection Experts generally agree there are four ways to combat deepfakes:

(1) Legislation and regulation. (2) Corporate polices and voluntary action.

(3) Education and training. (4) Anti-deepfake technology that includes

deepfake detection, content authentication and deepfake prevention.

Deepfake tools are becoming more efficient, however, there still are limitations in current logarithms as they attempt to reconstruct facial features. Detection methods are becoming better and frequently look at eye blinks since there are very few images available that include a subject with closed eyes. Other subtle defects include face warping and an unnatural skin texture. Some detection methods even go through two or more stages to modify an original image into something that can be manipulated better. However, even though there is not much research regarding the anti-deepfake effectiveness, blockchains may prove to be the best remedy for authentication. A blockchain will create a block of unique and unchangeable metadata. Smart contracts are also an explorable option with the integration of unique hashes (Nguyen, 2021). The number of eye blinks in deepfake videos tends to be a slower than in real life and eye color is particularly difficult to reproduce. Therefore, many deepfake detection tools focus on the eyes. Other telltale signs are variations in lighting that can leave unnatural shadows that reveal a video probably was altered. Convolutional neural networks (CNNs) work on machine learning and can be positioned at different locations throughout the internet to try to detect deepfake videos and alert others about their presence. Education and training are essential to combat deepfake technology, especially for the older

population that might be less technologically savvy. Some anti-deepfake technology provides tools to:

(1) Detect deepfakes. (2) Authenticate content. (3) Prevent content from being used to

produce deepfakes. However, there is far more available research and there are many more people attempting to create authentic looking deepfakes than there are those attempting to defeat deepfake technology. For example, the author reports: (1) Users upload 500 hours of content per minute on YouTube. (2) Twitter moderators attempt to screen 8 million accounts a week that attempt to spread content through manipulation. These are some quantitative challenges that merely scratch the surface of the emerging problem. There just are not enough screeners and the anti- deepfake technology experts are hopelessly outnumbered by the legitimate and illegitimate deepfake creators.

However, some media forensic experts suggested subtle indicators to detect deepfakes that included:

(1) Imperfections such as “face wobble,” shimmer and distortion.

(2) Wavy movements, inconsistent speech and mouth patterns.

(3) Abnormal movement of fixed objects. (4) Inconsistences in lighting, reflections and

shadows. (5) Blurred edges blurred facial features and

abnormal breathing. (6) Unnatural eye movement and overly

smooth skin. These and other inconsistencies give hope to the anti-deepfake activists who believe

automated procedures could detect deepfakes someday in a way that is similar to cellphone logarithms that recognize spam.

VI. Conclusion Deepfake technology is popular because it provides photorealistic results that can be created by an untrained layperson. However, there are experts and tools available to detect these fakes. The topic of deepfake technology is growing rapidly, cannot be ignored and requires some sort of regulation to improve authenticity for positive uses.

References [1] Mika Westerlund, The Emergence of Deepfake Technology: A Review, Technology Innovation Management Review, November 2019, https://timreview.ca/article/1282. [2] Thanh Thi Nguyen, et al, Deep Learning for Deepfakes Creating and Detection: A Survey, Cornell University, April 26, 2021, https://arxiv.org/abs/1909.11573. [3] Marwan Albahar and Jameel Almalki, Deepfakes: Threats and Countermeasures Systematic Review, Journal of Theoretical and Applied Information Technology, Nov. 30, 2019, http://www.jatit.org/volumes/Vol97No22/7V ol97No22.pdf.