Heart conditions and skin cancer in minorities could be missed by ‘biased’ tests
Heart conditions and skin cancer could be missed in women and ethnic minorities due to racially biased medical devices, a review has found.
An expert panel found that pulse oximeters – widely used during the Covid-19 pandemic to monitor blood oxygen levels – can overestimate the amount of oxygen in the blood of people with darker skin tones.
This could lead to delay in treatment if dangerously low oxygen levels were missed.
The Independent Review of Equity in Medical Devices also found evidence of potential biases against women, ethnic minority and disadvantaged socioeconomic groups in how clinical decision-making tools select higher risk patients needing more intensive treatment.
One example is the potential underdiagnosis of skin cancers for people with darker skin when using AI-enabled devices.
This is as a result of machines being “trained” predominantly on images of lighter skin tones. There is also a longstanding problem of underdiagnosis of cardiac condition in women, which Artificial Intelligence (AI) algorithms in medical devices could make worse, the review found.
For example, AI algorithms in many dermoscopes – microscope-like devices used by dermatologists to examine individual skin lesions more closely – may not cater as well for non-white skin.
“The consequences could include increased false negative error rates for skin cancer detection and delayed treatment for patients from some ethnic groups,” the review said.
Chair Professor Dame Margaret Whitehead, from the University of Liverpool, said that although the evidence for adverse clinical impacts of these biases is currently “patchy, though indicative”, the review made several recommendations focused on developing bias-free AI devices and medical devices.
It said the NHS and product makers need to engage with diverse groups of patients, patient organisations and the public when developing the devices.
The review also recommends that the Government should start preparing now for the disruption to healthcare from the next generation of AI-enabled machines if it is to minimise the risk of patient harm.
Professor Chris Holmes warned that the Government needs to understand how the advent of large language and foundation models like ChatGPT will disrupt clinical and public health practices.
He said: “We are calling on the Government to appoint an expert panel including clinical, technology and healthcare leaders, patient and public representatives and industry to assess the potential unintended consequences arising from the AI revolution in healthcare.
“Now is the time to seize the opportunity to incorporate action on equity in medical devices into the overarching global strategies on AI safety.”
Dame Margaret said: “The advance of AI in medical devices could bring great benefits, but it could also bring harm through inherent bias against certain groups in the population, notably women, people from ethnic minorities and disadvantaged socio-economic groups.
“Our review reveals how existing biases and injustices in society can unwittingly be incorporated at every stage of the lifecycle of AI-enabled medical devices, and then magnified in algorithm development and machine learning. Our recommendations therefore call for system-wide action by many stakeholders and now need to be implemented as a matter of priority with full government support.”
The review was commissioned in 2022 by the then Health Secretary Sajid Javid. He said: “The colour of someone’s skin or where they are from should not impact health outcomes. Yet, the pandemic highlighted how too many of these inequalities remain. I commissioned this review to help ensure every medical device, alongside the wider healthcare system, is delivering for every patient. I hope this review and its important recommendations will help deliver much-needed change.”
The Government said it accepted the review’s recommendations and would action to make the UK a “world leader in tackling ethnic and other biases in medical devices”.
A series of commitments include ensuring that pulse oximeter devices used in the NHS can be used safely across a range of skin tones, and removing racial bias from data sets used in clinical studies.
Ministers said some action has already been taken such as updating NHS guidance to highlight potential limitations of pulse oximeter devices on patients with darker skin tones.
Health Minister Andrew Stephenson said: “Making sure the healthcare system works for everyone, regardless of ethnicity, is paramount to our values as a nation. It supports our wider work to create a fairer and simpler NHS.”
Dr Sara Khalid, Associate Professor of Health Informatics and Biomedical Data Science at the University of Oxford, said: “This important review reinforces what experts have long-known about the role of health data poverty and resulting biased AI in worsening underlying systemic health inequities. It will be important to monitor if and how these practical recommendations influence real clinical practice.”
