Deepfake technology has been associated with a long list of dark, bizarre and slightly horrifying digital trends over recent years. Nonconsensual celebrity porn, dystopian political disinformation, and more have all been linked to the technology, giving it a controversial air. The emerging AI tool has even been used, in some cases, as a scapegoat—such as in that strange cheerleader vaping scandal we saw gain traction in 2021. The predominant worry that the general public has about deepfakes is that many are unable to detect whether something they encounter online is real or not. The Massachusetts Institute of Technology (MIT) has even created an ongoing research project which lets you know how well you can spot the difference. The difficulty of identifying what’s real as opposed to fabricated testifies to how believable and lifelike these computer-generated simulations can seem.

Another significant concern about deepfakes is their problematic nature when it comes to subverting truth without the consent of those in the images being transformed. The Guardian’s science editor, Ian Sample, defined deepfakes in January 2020 as being “the 21st century’s answer to Photoshopping.” Leveraging deep learning, a form of artificial intelligence, the technology is able to create fake events, primarily in the form of videos—though it can be applied to photos and audio as well.

Sample noted that many deepfakes found online are linked to porn. At the time of writing his article two years ago, up to 96 per cent of deepfakes found by AI firm DeepTrace involved simulation porn of female celebrities without their consent. This nonconsensual media is created by mapping the well-documented faces of celebrities onto the bodies of pornstars. While deepfakes related to women are primarily explicit content, DeepTrace’s report also revealed that men targeted by the technology are largely found on YouTube and the content they’re featured in tends to be commentary-based.

Though deepfakes seem to have only grown more controversial over the years, top-tier musicians have recently employed the technology in a different way—in the form of music videos. On 9 May 2022, days before the release of his album Mr. Morale & The Big Steppers, Kendrick Lamar dropped a music video for his single ‘The Heart Part 5’ on YouTube.

In the six-minute-long video, the award-winning rapper is seen from the chest up, wearing a simple white tee and bandana against a red backdrop. Over the course of the song, his face morphs into an array of famous black men, from OJ Simpson and Kanye West to Will Smith and Kobe Bryant. The most significant moment of the music video is perhaps when Lamar transforms into the late Nipsey Hussle and then raps from his perspective about his own murder from beyond the grave. Though uncanny at times, the film pushes the envelope in terms of artistic expression, allowing Lamar to vividly complement his lyrics with realistic depictions of other renowned black men.

Pitchfork senior staff writer Marc Hogan noted that “while the prospect of fake videos that seem legit has plenty of disturbing implications, it’s also a perfect tool for an artist who has long delighted in employing a range of voices in his work and destabilizing listeners’ concepts of identity.” By using emerging technology in a ‘wrong’—or rather innovative—way, Lamar was able to create groundbreaking visuals that further express his truth by using the visual language of the 21st century. Pitchfork writer, Dylan Green, observed that deepfakes, in this instance, “a​​mplify Lamar’s words and serve to visualize a complicated lineage through Blackness and the pressures of celebrity.”

In the same week, Kanye “Ye” West shared the music video for his track ‘Life of the Party’, which employed similar deepfake-styled clips. The music video featured an array of photos from the rapper’s childhood, each of which received new life—thanks to AI automation. Though the animation seems almost cartoon-like in comparison to the seamless verisimilitude of ‘The Heart Part 5’, both the videos demonstrate the wide range of artistic expression that deepfake technology can achieve, be it reinvigorating past voices and images à la Kanye, or drawing sharp comparisons between contemporaries à la Kendrick.

Deepfake technology, though synonymous with videos, can also closely imitate voices, fostering uneasy implications for its usage in the music industry. When the AI is employed in this way, it’s referred to as voice skins. Those whose voices are widely documented online—think podcasters, YouTubers, politicians and musicians—are more likely to be seamlessly replicated by the technology, as there’s more data (here, thousands of minutes of them speaking) to draw from.

Back in April 2020, audio files of Jay-Z rapping soliloquies from ‘Hamlet’ and Billy Joel’s ‘We Didn’t Start The Fire’ that nobody had ever heard before, surfaced online. A comment on the latter video succinctly described the realistic but fake audio clips as “entirely computer-generated using a text-to-speech model trained on the speech patterns of Jay-Z.” The videos were taken down at the request of Roc Nation, the entertainment agency founded by the American rapper, but then were put back up on the YouTube channel Voice Synthesis which features an array of high-profile voice skins, from past presidents to famous comedians. While the audio clips unmistakably sound like Jay-Z—though they aren’t a perfect replica—legal experts involved in the case don’t think any existing copyright law is being violated by deepfakes like these. In fact, many within the industry view this technology as a new form of sampling.

Deepfakes have had a fraught relationship with the music industry—rappers, in particular—over the recent years, as many navigate how to grapple with the possibilities of the technology. By reclaiming deepfakes in a creative fashion, Lamar and West have potentially pivoted their role in the music industry, giving new light to the otherwise malicious technology. Though the AI simulations still have disturbing implications in many ways, it will be interesting to witness the journey as it becomes yet another tool for artistic and musical expression. And, as deepfake technology continues to become more accessible to the average internet user, this synthetic music video style has even more potential to grow, for better or for worse.

Remember the website This Person Does Not Exist which, when visited, greeted browsers with the face of a stranger, one that would then completely change when the page was refreshed with a new stranger’s face? Those faces were  all deepfakes developed by algorithms, hence the name of the website. Since it appeared, there has been an influx in computer-generated images. The creator of This Person Does Not Exist, Phil Wang, made the site using a new development in machine learning called Generative Adversarial Networks (GAN). GAN forces two data sets to compete with each other in a game, encouraging each strand to learn from the other’s mistakes.

To train GAN, you feed it images of things you would like to generate more of, and it then simultaneously tries to generate images and learns how to distinguish its own images from the real ones (so to stay true to the original form or idea, in a sense).

As with many AI models, GANs require a large amount of data—the more data, the more variation. As a concept, this is both interesting and potentially useful, depending on what GAN is used for. However, collections of naked women that have been collected from porn production companies are being used to generate deepfake porn. The porn production companies involved in this new problematic doing have also previously been accused of forcefully coercing women to have sex on camera without their consent. Just so you can get an idea of the characters we’re dealing with here.

The dataset that is circulating in deepfake porn creation communities online includes images scraped from the porn production company Czech Casting in particular, which local police have accused of human trafficking and rape as well as trafficking still images from other porn sites. Czech Casting’s founder is currently a fugitive on the FBI’s most-wanted list.

Much like This Person Does Not Exist, the dataset is being used to create photorealistic images of women, but nude, who aren’t real and who all don’t look like any one person. Essentially, it’s porn generated entirely by AI: deepfakes porn, or deepnudes as we once named the same concept. Because of this, these algorithmically-generated images that are created aren’t technically doing much harm as they are not ‘real people’. However, legal experts, technologists and the women included in the datasets describe these creations as uniquely dehumanising.

Honza Červenka, a lawyer at the McAllister Olivarius law firm who specialises in revenge porn and technology has been following Czech Casting’s case. He told Vice that the idea of these images being less harmful because of being run through an algorithm makes them “anonymised” is a red herring. He said that “It’s mad science really, and completely and utterly re-victimizing to the victims of the Czech Casting perpetrators.”

The Czech Casting case does not stand alone, as it is simply impossible for technology to reverse in its tracks, meaning that this technology will undoubtedly develop. For now, GAN is incapable of generating videos similar to real porn videos, the best it can do is generate images. Going forward, there are several issues associated with algorithmic-generated porn. In the long run though, the difference between real and computer-generated videos will shrink until AI-generated porn potentially becomes mainstream.

Studio-made pornography may still exist, but it might become more of a niche interest for people who want to watch ‘real people have real sex’ instead of computer-generated videos. Recruitment agencies and production companies might start to lean towards companies that are involved in this AI-generated pornography and possibly provide them with data.

A huge issue with the algorithms used to create this kind of porn is that because programming is variable, the technology has the worrying potential of being used to target people, within any criteria. The algorithm can only produce things that it has seen before though, as this is how GAN generates data, however, the potential remains.

Computer scientist and co-founder of SketchDeck, David Mack, attempted to build an AI that generated porn and wrote an article on his experience. In it, he reflected on the fact that on a macro scale, if his project were a success, it had the potential to change the world. “Many people are harmed in the production of pornography, and this project (given the very low cost of producing new images) would supplant them. Pornography would harm fewer people.”

However on the other side, he wrote that “this would displace workers and reshape an industry.” He concluded that he had struck a corner of hypocrisy within our society and technology. Pornography is a major part of the internet’s usage and people’s daily lives, this new technology could potentially improve the darker and more dangerous sides to the porn industry, such as sexual abuse and trafficking.

All in all, there are benefits to the idea, but while the porn industry abuses the women within it, like in the Czech Casting case among others, the negatives outride the benefits—without a fully ethical support system to generate the data for the project, its future remains unclear. One thing we know for sure is that this tech and the industries it circles within will move fast.