Twitter’s image cropping algorithm favours “slim, beautiful and light-skinned faces”

By Malavika Pradeep

Published Aug 11, 2021 at 02:00 PM

Reading time: 2 minutes

On 19 September 2020, PhD student Colin Madland posted a Twitter thread with images of himself and a black colleague—who had been erased from a Zoom call after its algorithm failed to recognise his face. When Madland viewed the tweet on his phone, Twitter chose to crop his colleague out of the picture altogether. The findings triggered several accusations of bias as Twitter users published photos to analyse whether the AI would choose the face of a white person over a black person or if it would focus on women’s chests over their faces.

Twitter has been automatically cropping images to prevent them taking up too much space on the main feed and to allow multiple pictures to be shown in the same tweet. Dubbed the “saliency algorithm,” it decides how images would be cropped in Twitter previews, before being clicked to open at full size. But when two faces were in the same image, users discovered how the preview appeared to favour white faces, hiding the black faces until users clicked on the image.

https://twitter.com/_jsimonovski/status/1307542747197239296

Two days later, Twitter apologised for its ‘racist’ image cropping algorithm with a spokesperson admitting that the company had “work to do.” “Our team did test for bias before shipping the model and did not find evidence of racial or gender bias in our testing,” the spokesperson said. “But it’s clear from these examples that we’ve got more analysis to do. We’ll continue to share what we learn, what actions we take, and will open source our analysis so others can review and replicate.” The company thereby disabled the system in March 2021.

On 19 May 2021, Twitter’s own researchers analysed the algorithm and found a very ‘mild’ racial and sexist bias. As an attempt to open source and analyse the problem more closely, however, Twitter held an “algorithmic bug bounty”—an open competition held at the DEFCON security conference in Las Vegas. Findings from the competition embarrassingly confirmed the earlier allegations.

The competition’s first place entry—and winner of the top $3,500 prize—was Bogdan Kulynych, a graduate student at EPFL, a research university in Switzerland. Using an AI program called StyleGAN2, Kulynych generated a large number of realistic faces which he varied by skin color, feminine versus masculine facial features and slimness. He then fed these variants into Twitter’s image-cropping algorithm to analyse the ones it preferred.

The results? The algorithm preferred faces that are “slim, young, of light or warm skin color and smooth skin texture, with stereotypically feminine facial traits.” It doesn’t stop there. The second and third-placed entries showed that the system was biased against people with white or grey hair—suggesting age discrimination—and favouring English over Arabic script in images.

According to Kulynych, these algorithmic biases amplify biases in society, thereby cropping out “those who do not meet the algorithm’s preferences of body weight, age, skin color.”

“Algorithmic harms are not only ‘bugs’,” he wrote on Twitter. “Crucially, a lot of harmful tech is harmful not because of accidents, unintended mistakes, but rather by design.” Kulynych also noted how these biases come from the maximisation of engagement and profits in general.

The results of Twitter’s “algorithmic bug bounty” calls upon the need to address societal bias in algorithmic systems. It also shows how tech companies can combat these problems by opening their systems up to external scrutiny. And now we have our eyes trained on you, Facebook, and your gender-biased job advertisements.

Keep On Reading

By Abby Amoakuh

Is football apolitical? Here is how FIFA and the UEFA are used to further political agendas

By Fleurine Tideman

Is BeReal dead? We asked two social media experts and the app’s COO to find out

By Abby Amoakuh

Sydney Sweeney sex tape leak malware used as bait by hackers on Twitter

By Fatou Ferraro Mboup

From viral Boiler Room sessions to Ibiza residencies, DJ duo Prospa are only getting started

By Charlie Sawyer

Russian scientist injects himself with 3.5-million-year-old bacteria to try and live forever

By Abby Amoakuh

Selena Gomez haters use singer’s comments on Israel-Hamas war to reignite Hailey Bieber feud

By Alma Fabiani

Woman sues Lyft alleging driver repeatedly raped her and impregnated her

By Abby Amoakuh

Alabama Barker denies claims she has had a lot of plastic surgery in major clapback

By Charlie Sawyer

Florida plans to expand Ron DeSantis’ Don’t Say Gay law into workplaces and ban use of preferred pronouns

By Emma O'Regan-Reidy

From gen Z farming to pro-hybrid work, here are 3 ways the younger generation will impact 2024

By Abby Amoakuh

What does rizz mean? Learn why it’s Oxford’s Word of the Year for 2023

By Alma Fabiani

Rebel Wilson reveals member of Royal family invited her to lose virginity in drug-fuelled orgy

By Charlie Sawyer

Megan Fox accused of xenophobia after comparing bad photo of herself to Ukrainian blowup doll

By Fatou Ferraro Mboup

Forget about bed bugs, a Scabies outbreak is already sweeping the UK

By Fatou Ferraro Mboup

After becoming Elvis Presley, Austin Butler reveals why he couldn’t do method acting for Dune: Part 2

By Abby Amoakuh

Trump kept a book of Adolf Hitler’s speeches, Ivana Trump reveals

By Charlie Sawyer

OnlyFans models are using breastfeeding content as a loophole to bypass Instagram’s nudity policy

By Fatou Ferraro Mboup

US politics this week: Biden continues to use TikTok after signing ban and Trump rises among Black voters

By Abby Amoakuh

Shocking list of F1 legends who have been linked to Jeffrey Epstein in latest court documents

By Charlie Sawyer

Zac Efron reveals gruelling body transformation for upcoming wrestling film The Iron Claw