Twitter’s image cropping algorithm favours “slim, beautiful and light-skinned faces”

By Malavika Pradeep

Published Aug 11, 2021 at 02:00 PM

Reading time: 2 minutes

On 19 September 2020, PhD student Colin Madland posted a Twitter thread with images of himself and a black colleague—who had been erased from a Zoom call after its algorithm failed to recognise his face. When Madland viewed the tweet on his phone, Twitter chose to crop his colleague out of the picture altogether. The findings triggered several accusations of bias as Twitter users published photos to analyse whether the AI would choose the face of a white person over a black person or if it would focus on women’s chests over their faces.

Twitter has been automatically cropping images to prevent them taking up too much space on the main feed and to allow multiple pictures to be shown in the same tweet. Dubbed the “saliency algorithm,” it decides how images would be cropped in Twitter previews, before being clicked to open at full size. But when two faces were in the same image, users discovered how the preview appeared to favour white faces, hiding the black faces until users clicked on the image.

Two days later, Twitter apologised for its ‘racist’ image cropping algorithm with a spokesperson admitting that the company had “work to do.” “Our team did test for bias before shipping the model and did not find evidence of racial or gender bias in our testing,” the spokesperson said. “But it’s clear from these examples that we’ve got more analysis to do. We’ll continue to share what we learn, what actions we take, and will open source our analysis so others can review and replicate.” The company thereby disabled the system in March 2021.

On 19 May 2021, Twitter’s own researchers analysed the algorithm and found a very ‘mild’ racial and sexist bias. As an attempt to open source and analyse the problem more closely, however, Twitter held an “algorithmic bug bounty”—an open competition held at the DEFCON security conference in Las Vegas. Findings from the competition embarrassingly confirmed the earlier allegations.

The competition’s first place entry—and winner of the top $3,500 prize—was Bogdan Kulynych, a graduate student at EPFL, a research university in Switzerland. Using an AI program called StyleGAN2, Kulynych generated a large number of realistic faces which he varied by skin color, feminine versus masculine facial features and slimness. He then fed these variants into Twitter’s image-cropping algorithm to analyse the ones it preferred.

The results? The algorithm preferred faces that are “slim, young, of light or warm skin color and smooth skin texture, with stereotypically feminine facial traits.” It doesn’t stop there. The second and third-placed entries showed that the system was biased against people with white or grey hair—suggesting age discrimination—and favouring English over Arabic script in images.

According to Kulynych, these algorithmic biases amplify biases in society, thereby cropping out “those who do not meet the algorithm’s preferences of body weight, age, skin color.”

“Algorithmic harms are not only ‘bugs’,” he wrote on Twitter. “Crucially, a lot of harmful tech is harmful not because of accidents, unintended mistakes, but rather by design.” Kulynych also noted how these biases come from the maximisation of engagement and profits in general.

The results of Twitter’s “algorithmic bug bounty” calls upon the need to address societal bias in algorithmic systems. It also shows how tech companies can combat these problems by opening their systems up to external scrutiny. And now we have our eyes trained on you, Facebook, and your gender-biased job advertisements.

Keep On Reading

By Fatou Ferraro Mboup

What’s the happiest country in the world? Spoiler: it didn’t win fairly

By Abby Amoakuh

Khloé Kardashian blasts Kris Jenner over cheating scandal in episode 4 of The Kardashians

By Phoebe Dodds

Are you in the middle of a quarter-life crisis? Here’s how to shake it off

By Charlie Sawyer

Hair today, bare tomorrow: Nudist barbershops are trending in Brazil

By Fatou Ferraro Mboup

Understanding the Carla Foster case and how it might impact abortion rulings in the UK

By Charlie Sawyer

Scientists discover terrifying new sea species that looks exactly like the iconic monster from Alien

By Charlie Sawyer

Watch this video of Luigi ruining a wedding proposal at Super Nintendo World

By Fatou Ferraro Mboup

French riots fuel far-right National Rally’s march into the mainstream. Why is no one reporting on it?

By Charlie Sawyer

Understanding the sneaky link: Meaning of the dating trend beyond the hook-up

By Charlie Sawyer

TikToker reveals the underwhelming food served at the White House Pride event

By Fatou Ferraro Mboup

What is legal cocaine? And how is it now being incorporated into our food and drinks?

By Charlie Sawyer

With Amy Schumer on the verge of being cancelled, here are 4 of her most problematic moments

By Jennifer Raymont

Zendaya’s Challengers is making tenniscore the trend of the summer, get us to Wimbledon now

By Fatou Ferraro Mboup

US town bans autumn-obsessed influencers from taking pictures of its fall foliage

By Jennifer Raymont

From the foodie fashion girlie to The Bear’s Carmy, we’ve all got the hots for chefs

By Charlie Sawyer

New cancer-killing pill could completely eradicate tumours from patients’ bodies

By Alma Fabiani

Exploring AI trends: Advancements, impact, and future possibilities

By Charlie Sawyer

Who are nepo baby Matty Healy’s parents? Meet Denise Welch and Tim Healy

By Fatou Ferraro Mboup

Polish TV show faces backlash over white performer in blackface impersonating Kendrick Lamar

By Charlie Sawyer

My chat with stand-up comedian Stef Dag, the Hot & Single host getting people laid in NYC