New research sheds light on how AI stacks up against human decision-making in peronsalised cancer treatment.
Treating cancer is becoming increasingly complex, but also offers more possibilities than ever. AI is expected to play a critical role in advancing the field, assisting with the laborious and time-consuming data analysis and interpretation that is required for personalised therapies.
But how does it compare to human decision-making?
Researchers at Charité – Universitätsmedizin Berlin and Humboldt-Universität zu Berlin have studied how effective generative artificial intelligence (AI) tools such as ChatGPT can be in this process.
If the body can no longer repair certain genetic mutations, cells begin to grow unchecked, producing a tumour. The crucial factor in this phenomenon is an imbalance of growth-inducing and growth-inhibiting factors, which can result from changes in oncogenes – genes with the potential to cause cancer – for example.
Precision oncology, a specialised field of personalised medicine, leverages this knowledge by using specific treatments such as low-molecular-weight inhibitors and antibodies to target and disable hyperactive oncogenes.
The first step in identifying which genetic mutations are potential targets for treatment is to analyse the genetic makeup of the tumour tissue.
The molecular variants of the tumour DNA that are necessary for precision diagnosis and treatment are determined, then the doctors use this information to craft individual treatment recommendations.
In especially complex cases, this requires knowledge from various fields of medicine. At Charité, this is when the “molecular tumour board” (MTB) meets: Experts from the fields of pathology, molecular pathology, oncology, human genetics and bioinformatics work together to analyse which treatments seem most promising based on the latest studies.
Can artificial intelligence help with treatment decisions?
The researchers wondered whether artificial intelligence might be able to help at this juncture.
In a study published in the journal JAMA Network Open, they examined the possibilities and limitations of large language models such as ChatGPT in automatically scanning scientific literature and selecting personalised treatments.
“We prompted the models to identify personalized treatment options for fictitious cancer patients and then compared the results with the recommendations made by experts,” Dr Damian Rieke, a doctor at Charité, said. “AI models were able to identify personalised treatment options in principle – but they weren’t even close to the abilities of human experts.”
The team created ten molecular tumour profiles of fictitious patients for the experiment. A human physician specialist and four large language models were then tasked with identifying a personalised treatment option. These results were presented to the members of the MTB for assessment, without them knowing where which recommendation came from.
“There were some surprisingly good treatment options identified by AI in isolated cases,” Benary reports. “But large language models perform much worse than human experts.” Beyond that, data protection, privacy, and reproducibility pose particular challenges in relation to the use of artificial intelligence with real-world patients, she notes.
Still, Rieke is fundamentally optimistic about the potential uses of AI in medicine. He said; “In the study, we also showed that the performance of AI models is continuing to improve as the models advance. This could mean that AI can provide more support for even complex diagnostic and treatment processes in the future – as long as humans are the ones to check the results generated by AI and have the final say about treatment.
Professor Felix Balzer, director of the Institute of Medical Informatics, is also certain that medicine will benefit from AI.
“One special area of focus when it comes to greater efficiency in patient care is digitalization, which also means the use of automation and artificial intelligence,” Balzer explains.
His institute is working on AI models to help with fall prevention in long-term care, for example.
Other areas at Charité are also conducting extensive research on AI. The Charité Lab for Artificial Intelligence in Medicine is working to develop tools for AI-based prognosis following strokes and the TEF-Health project is working to facilitate the validation and certification of AI and robotics in medical devices.
