Everyday, we learn more about how AI is being used in new and innovative ways to combat serious issues. One department that can’t seem to get enough of the technology is the medical world. From being used to diagnose early signs of dementia and helping paralysed people write with their minds to something as ridiculously funny as smart toilets protecting your health, AI is set to be the future of healthcare. Perfect, right? Well, not quite.

Much like most infrastructures and systems, AI is no stranger to the systemic racism that permeates through almost all institutions of our current society. While there are examples of its use to actually combat racism—take Sajid Javid’s drive to utilise AI in tackling racial inequality throughout the NHS—the system is not without its faults. In fact, its use in recent years has shown a historic pattern of discrimination, from Facebook’s (now Meta) AI labelling black men as ‘primates’ to failing to distinguish different people of colour from each other, the tech has a long way to go. In fact, its projected saturation in society has led to real fears that AI could soon ‘hack’ our biometric data.

Now, another supposed breakthrough in AI medicine comes with its obvious catch; new research has surfaced that details how AI systems being trained to diagnose skin cancer is far less effective on darker skin.

The new AI recognition technology works by training the systems to develop machine learning algorithms that can identify and diagnose specific types of skin cancer, The Guardian reports. This method of diagnosis reportedly matches the success rate of diagnosis from human medical professionals. However, while this could revolutionise this sector of healthcare, it would also represent a risk of certain groups and demographics being left behind.

Researchers are arguing for significant strides to be made which would assure all patients that they can benefit from these technological advancements. Like much of the medical world, the skin image databases that could be used as part of the AI’s ‘training’ to identify skin cancer contain very few, if any, examples of darker skin. This leaves out vital information needed for the diagnosis pertaining to different ethnic groups and skin types. In simpler terms, the AI’s effectiveness in diagnosis depends largely on the quality of the data it is trained with. This important research, and its findings, were first presented at the National Cancer Research Institute (NCRI) Festival—a national cancer conference—and published in Lancet Digital Health.

Presented by Doctor David Wen from the University of Oxford, the first author of the study, Wen stated, “AI programs hold a lot of potential for diagnosing skin cancer because it can look at pictures and quickly and cost-effectively evaluate any worrying spots on the skin. However, it’s important to know about the images and patients used to develop programs, as these influence which groups of people the programs will be most effective for in real-life settings.” Wen went on to highlight how this would breed an exclusionary medical method in the field.

“You could have a situation where the regulatory authorities say that because this algorithm has only been trained on images in fair-skinned people, you’re only allowed to use it for fair-skinned individuals, and therefore that could lead to certain populations being excluded from algorithms that are approved for clinical use,” he explained. And even if you were allowed to use it, it most likely wouldn’t be effective, “Alternatively, if the regulators are a bit more relaxed and say: ‘OK, you can use it [on all patients]’, the algorithms may not perform as accurately on populations who don’t have that many images involved in training.”

This could bring a swathe of complications such as failing to spot treatable cancers and accurately assessing the risks of surgery, the researchers explained. Their comprehensive study, published in Lancet Digital Health, details how the scientists utilised a “combined MEDLINE, Google and Google Dataset search” that found 21 open-access skin cancer databases containing over 100,000 images. The percentage of which that includes ethnic groups is disheartening.

The researchers recorded that only a few of these 21 datasets noted the ethnic group or skin type to which the individuals photographed belong. Among the 106,950 images, only 2,436 had a skin type detailed along the image. Of these 2,436, the total number of images that recorded the individual as having brown skin was ten—-with only one being listed as having dark brown or black skin.

“We found that for the majority of datasets, lots of important information about the images and patients in these datasets wasn’t reported. There was limited information on who, how and why the images were taken. […] This can potentially lead to the exclusion or even harm of these groups from AI technologies,” Wen added.

Such failings could quite literally be a matter of life or death for people of colour, Wen further explained, “Although skin cancer is rarer in people with darker skins, there is evidence that those who do develop it may have worse disease or be more likely to die of the disease. One factor contributing to this could be the result of skin cancer being diagnosed too late.” In order to combat this, the team of researchers hope to use their data to fuel the creation of health data quality standards to be implemented in the future and continuous development of AI in medical care. Much like the drive Javid proposed, the standards will include specific requirements in the representation of certain patients and cite the individuals that must be recorded.

Hopefully, this step becomes one of many that sets out to equalise medical health information and better aid the lives of everyone.

Artificial intelligence (AI) is continuously making grounds in a multitude of industries, one being the medical world. First used to diagnose early onset dementia, it will now aim to tackle racial inequalities in medicine as part of Sajid Javid’s—the UK’s health secretary—plan to “level up” the country’s health service. A new report by The Guardian details the projects behind this new governmental initiative.

Javid described his reasons behind the AI initiative to the publication, “As the first health and social care secretary from an ethnic minority background, I care deeply about tackling the disparities which exist within the healthcare system. As we recover from the pandemic we have an opportunity for change, to level up, and ensure our NHS is meeting the needs of everyone.”

The past 18 months have only continued to shed a light on the ever-present widespread issue of racial inequality in many pillars of society—one being the medical field. Structural racism in healthcare leads only to poorer health conditions for those from black, Asian and ethnic minority backgrounds. A 2020 Sky News report on such matters found that black women are five times more likely to die in childbirth than white women in the UK. Structural racism impacts even medical equipment, with a recent review carried out by the NHS Race and Health Observatory finding that pulse oxygen monitors may be less accurate on darker skin.

These problems have been present for decades, and the COVID-19 pandemic has only exacerbated them. Speaking on this particular inequality, Javid stated, “It is unacceptable that black women in England are five times more likely to die from complications during childbirth than their white counterparts. AI could help us better understand why this is the case and ensure black mothers have an equal chance for a healthy life with their newborn.”

The above issue is behind one of the projects put forward as part of this initiative; the project will employ computer algorithms to study and examine the causes behind such maternity issues for black mothers. The findings from such research will then be implemented as part of updated training for nurses and midwives.

This AI effort to clamp down on racism in health care, titled the AI Ethics Initiative, will be fronted by NHSX—a governmental department that focuses on improving NHS policy, data and technology. Other potential projects include the creation of an AI-powered chatbot tasked with aiding the screening of sexually-transmitted infections (STIs) and HIV among ethnic minority groups. Another AI screening project in this drive aims to better identify and diagnose diabetic retinopathy in such communities.

Head of AI research and ethics at NHSX, Brhmie Balaram, told The Guardian, “Artificial intelligence has the potential to revolutionise care for patients, and we are committed to ensuring that this potential is realised for all patients by accounting for the healthy needs of diverse communities.” One of the most critical AI projects in this new goal is the updating of UK health data that would more accurately represent the country’s population. Currently, there seems to be a great divide in data which should be more inclusive.

For Javid, this is the first most important and integral part of the drive, “If we only train our AI using mostly data from white patients it cannot help our population as a whole. We need to make sure the data we collect is representative of our nation. This new funding will support the development of a much-needed set of standards to make sure datasets for training and testing AI systems are diverse and inclusive so no one is disadvantaged because of their race.”

While entrusting AI comes with its own issues—take Facebook’s AI algorithm labelling black men as ‘primates’—he continued by highlighting the positives of such technology, “Technology, particularly AI, can be an incredible force for good. It can save valuable clinician time and help provide faster, more accurate diagnosis, so patients can access the care they need as quickly as possible. It can also help us better understand racial differences so we can train our workforce to look for different symptoms or complicating factors, diagnose faster, and tailor treatments.”

As we hopefully await potential improvements coming from these projects, we can’t help but wonder if it’s too good to be true. With claims of voter suppression, an attack on immigrants, a war on Black Lives Matter and still legal revirginisation surgery—to name a few—the UK has a long way to go in the battle for equality.