When biomedical engineering student Tomisin Adebari scanned her own skin condition into an artificial intelligence system, she expected the technology to provide insight. Instead, the AI tool gave her an incorrect diagnosis. The experience raised a larger question about the growing role of artificial intelligence in healthcare: If AI systems are trained on incomplete information, can they accurately serve everyone?
Artificial intelligence is becoming a part of modern medicine. In dermatology, which heavily relies on examining images, artificial intelligence tools are being made to help doctors find skin cancer, infections and other diseases. By analyzing thousands of medical images, these systems have the potential to assist physicians, improve early detection and expand access to healthcare.
However, researchers are finding out that these technologies do not work the same for all patients. When artificial intelligence systems are trained using datasets that lack diversity, they can struggle to accurately recognize medical conditions in underrepresented groups, including people with darker skin tones. As artificial intelligence becomes more common in healthcare, experts are wondering if these tools are accidentally repeating the racial disparities that have been in medicine for generations.
One major concern surrounding artificial intelligence is the data used to train these systems. Artificial intelligence does not learn like a doctor. It looks for patterns in collections of data, so the quality and diversity of the data it uses directly affect how well it works. If some groups of people are missing from that data, AI may become less accurate when analyzing patients from those groups.
A study in 2024 looked at how artificial intelligence helped doctors diagnose skin conditions from medical images. The study found that artificial intelligence improved physicians’ overall accuracy, especially for primary care doctors who may have less specialized experience in dermatology. However, researchers also discovered that the improvement was not equal across different skin tones.
When doctors used AI to help them diagnose conditions on lighter skin, they were more accurate than when they diagnosed conditions on darker skin. Researchers suggested that one possible explanation is the lack of representation of darker skin tones in medical education and training materials. When AI systems are developed using limited examples, they can unintentionally continue those same gaps.
This issue is further demonstrated in research published in Science Advances by Roxana Daneshjou and other researchers. The study analyzed how several dermatology AI systems performed when tested with a more diverse collection of clinical images. The researchers created the Diverse Dermatology Images (DDI) dataset, which included hundreds of confirmed medical images representing a wider range of skin tones.
The results showed that some AI systems performed significantly worse when evaluated with more diverse images than with the datasets they were originally trained on. For example, one system’s accuracy dropped when tested on the DDI dataset compared to its original evaluation data. Researchers found similar differences when comparing diagnoses between lighter and darker skin tones.
The study also showed one possible way to improve AI systems: making the data used to train artificial intelligence more diverse. When researchers used representative pictures to train the systems, the gap in performance between different skin tones became smaller. This shows that the problem is not that artificial intelligence cannot work in dermatology, but that it needs to be made with data that represents all patients.
The problems with AI bias are not just in dermatology. Tomisin Adebari discussed this in a TEDx Talk, where she said that healthcare technology often reflects the inequalities that are already in medical research. She mentioned how some medical devices, like pulse oximeters, have not worked as well for Black patients because the data and testing behind those devices did not include enough diverse participants.
According to Adebari, artificial intelligence is only as good as the data it is given. If the data used to make artificial intelligence systems is incomplete or biased, the technology will have the same problems. While artificial intelligence can process information quickly, it cannot fix the problems in the data unless humans make an effort to improve it.
For African Americans and other communities of color, this issue is connected to a history of unequal healthcare experiences. Medical research has not always included diverse populations equally, and those gaps can influence the development of new technologies. As healthcare becomes more dependent on artificial intelligence, researchers argue that representation must become a priority.
Despite these concerns, experts do not believe AI should be abandoned in healthcare. Instead, many researchers see it as a tool if it is developed carefully. Making medical data more diverse, including participants in research and testing algorithms regularly can help create systems that benefit more patients.
Artificial intelligence has the potential to transform healthcare by helping doctors make faster and more accurate decisions. However, technology alone cannot create fairness. As AI continues entering hospitals and medical offices, researchers say the goal should not only be creating smarter systems, but creating systems that work for everyone.
The future of healthcare AI may depend on one important question: Who is included in the data?
