The dangers of deepfake and AI technology seem to be constantly on the rise. From gross and harmful exploitation of people’s likeness for sexually explicit imagery to artists facing copyright nightmares over stolen artwork that was generated through machine learning, AI is constantly spawning new risks and problems for society. And the latest viral news that a TikTok advertisement is using artificial recreations of celebrities to promote products is just the tip of the iceberg.

In the latest AI catastrophe to hit the internet, a viral advert sees controversial podcast host Joe Rogan trying to sell you alleged libido-boosting supplements. The clip uses actual footage from an episode of The Joe Rogan Experience which has been edited using deepfake technology in order to make it look like Rogan is actually praising the benefits of “that Alpha Grind product.” An AI voice dubbing tool further creates the auditory illusion that the American commentator said these words. The result is chillingly convincing.

BREAKING: This AI deepfake video rendering of Joe Rogan is going viral on TikTok. This is the start of massive waves of new scams & misinformation!

If only there was a decentralized GPU compute/rendering network that can authenticate provenance for IP. $RNDR https://t.co/m5WQvJjEuF pic.twitter.com/VqtQgEFcyJ

— M A C H I N E 4 L P H A ⭕️ (@MACHINE4LPHA) February 13, 2023

Rogan pulls in an average of 11 million listeners per episode. So, a product endorsement from him is likely going to boost your exposure and net you a nice rise in sales. However, in order to get a hold of that publicity, brands have previously had to fork out gigantic fees. This is no longer an issue when you consider the capabilities of AI.

While the dubbed videos aren’t completely perfect, it’s easy to see how an absent-minded viewer might fall for this scam—believing that the product being sold is endorsed by a celebrity they trust, or who might have authority on the goods in question.

https://twitter.com/jpotts_48/status/1599919246964060161?s=20

As users began flooding Twitter with their surprise and shock at the false advert, one user perfectly captured some of the serious issues surrounding the viral moment, reminding us how dangerous deepfake technology has been for women for years. It’s only when a hyper-masculine man such as Rogan is used or manipulated that other men take note.

Men finally mad about deepfakes because they used their precious Joe Rogan’s image to sell shitty supplements on Tik Tok. I wonder if they would be anywhere near as mad if it was a deep fake of Joe being pegged? Guess that would’ve been fine BUT TO SELL STUFF NO WAY THATS FUCKED!

— FEjane (@mad__deity) February 14, 2023

According to Mashable, the original posting of the ad has since been removed from TikTok and the originally circulating tweet that housed the viral clip has since been disabled due to a complaint from “the copyright owner.”

The clips were removed from the video-sharing platform as they violated the platform’s “harmful misinformation policy.” The account that initially posted the ads was also banned. So, “what’s the issue?” one Twitter user asks. Professor of Neurobiology at Stanford, Andrew D Huberman(the other guest from the original clip) replied saying: “They created a false conversation. We never had. We were talking about something very different.”

What is being highlighted in these adverts is the very real danger that deepfakes such as this presents. There is an inherent ethical problem with content that breaks boundaries of consent, image rights and consumer trust.

Misinformation and falsehoods are easily recreated too thanks to the machine learning technology and the possibilities are, well, terrifying. First it’s fake celebrity endorsements, next it’s political speeches and deepfake cover-ups.

While the fake endorsement may seem tame, we’re on a very scary path,one where we need more opportunity to accurately verify the authenticity of videos like this, especially as they continue to get more and more realistic. Don’t even get me started on the threat that AI poses to employment opportunities for artists and creators.

I’m glad that the social media platforms housing the content acted fast and doled out significant punishments to the offenders. It’s strange knowing that it’s up to Silicon Valley to ensure that stuff like this doesn’t get normalised. I don’t want to live in a world where I can’t verify that the person I’m listening to is real. Do you?