Twitter’s image cropping algorithm favours “slim, beautiful and light-skinned faces”
On 19 September 2020, PhD student Colin Madland posted a Twitter thread with images of himself and a black colleague—who had been erased from a Zoom call after its algorithm failed to recognise his face. When Madland viewed the tweet on his phone, Twitter chose to crop his colleague out of the picture altogether. The findings triggered several accusations of bias as Twitter users published photos to analyse whether the AI would choose the face of a white person over a black person or if it would focus on women’s chests over their faces.
Twitter has been automatically cropping images to prevent them taking up too much space on the main feed and to allow multiple pictures to be shown in the same tweet. Dubbed the “saliency algorithm,” it decides how images would be cropped in Twitter previews, before being clicked to open at full size. But when two faces were in the same image, users discovered how the preview appeared to favour white faces, hiding the black faces until users clicked on the image.
Flipped the image....@Twitter is trash. pic.twitter.com/GxlNIEryFD
— Colin Madland 🇺🇦 (@colinmadland) September 19, 2020
I wonder if Twitter does this to fictional characters too.
— [email protected] (@_jsimonovski) September 20, 2020
Lenny Carl pic.twitter.com/fmJMWkkYEf
Two days later, Twitter apologised for its ‘racist’ image cropping algorithm with a spokesperson admitting that the company had “work to do.” “Our team did test for bias before shipping the model and did not find evidence of racial or gender bias in our testing,” the spokesperson said. “But it’s clear from these examples that we’ve got more analysis to do. We’ll continue to share what we learn, what actions we take, and will open source our analysis so others can review and replicate.” The company thereby disabled the system in March 2021.
On 19 May 2021, Twitter’s own researchers analysed the algorithm and found a very ‘mild’ racial and sexist bias. As an attempt to open source and analyse the problem more closely, however, Twitter held an “algorithmic bug bounty”—an open competition held at the DEFCON security conference in Las Vegas. Findings from the competition embarrassingly confirmed the earlier allegations.
The competition’s first place entry—and winner of the top $3,500 prize—was Bogdan Kulynych, a graduate student at EPFL, a research university in Switzerland. Using an AI program called StyleGAN2, Kulynych generated a large number of realistic faces which he varied by skin color, feminine versus masculine facial features and slimness. He then fed these variants into Twitter’s image-cropping algorithm to analyse the ones it preferred.
The results? The algorithm preferred faces that are “slim, young, of light or warm skin color and smooth skin texture, with stereotypically feminine facial traits.” It doesn’t stop there. The second and third-placed entries showed that the system was biased against people with white or grey hair—suggesting age discrimination—and favouring English over Arabic script in images.
According to Kulynych, these algorithmic biases amplify biases in society, thereby cropping out “those who do not meet the algorithm’s preferences of body weight, age, skin color.”
“Algorithmic harms are not only ‘bugs’,” he wrote on Twitter. “Crucially, a lot of harmful tech is harmful not because of accidents, unintended mistakes, but rather by design.” Kulynych also noted how these biases come from the maximisation of engagement and profits in general.
The results of Twitter’s “algorithmic bug bounty” calls upon the need to address societal bias in algorithmic systems. It also shows how tech companies can combat these problems by opening their systems up to external scrutiny. And now we have our eyes trained on you, Facebook, and your gender-biased job advertisements.
