Tensions Surrounding AI Concerns and Ethical Considerations
From a legal and ethical perspective, Firth-Butterfield highlighted issues of persistent bias and access. “How do we think about fairness, accountability? Who do you sue when something goes wrong? Is there somebody to sue?” she asked.
She also questioned the kind of data shared with generative AI systems, and brought attention to recent news of Samsung employees unintentionally leaking confidential information to ChatGPT. “That’s the sort of thing that you are going to have to be thinking about very carefully as we begin to use these systems,” she said.
Last month, Firth-Butterfield signed an open letter calling for a six-month pause on the development of AI systems “more powerful than GPT-4.” She decided to sign the letter because she said it was important to think deeply about this next, major step in AI development.
“What worries me is that we are hurtling into the future without actually taking a step back and designing it for ourselves,” Firth-Butterfield said.
DIVE DEEPER: Learn about Banner Health’s unified data model journey.
She stressed the importance in defining the problem and improving public understanding of AI. “What is it that we want from these tools for our future, and to make that really equitable?” she asked. “How do we design a future that enables everybody to access these tools? That’s why I signed the letter.”
Blackman raised questions about the black-box nature of AI models and characterized tools such as GPT-4 as “a word predictor, not a deliberator.”
“What’s the appropriate benchmark for safe deployment?” Blackman asked. “If you’re making a cancer diagnosis, I need to understand exactly the reasons why you’re giving me this diagnosis.”
Lee pushed against Blackman’s perspective, suggesting that the black-box issue might not exist at some point in future development, and that the “word predictor” description oversimplifies complex processes.
Ultimately, Blackman said, people should push for enterprisewide governance over AI, not to stop innovation but to establish a way to systematically assess the risks and opportunities on a use case basis. If not, things will fall between the cracks, he said, and possibly cause great harm.
“You need certain kinds of oversight. It can’t just be the data scientists,” he added. “It needs to be a cross-functional team. There are legal risks, ethical risks, risks to human rights, and if you don’t have the right experts involved in thinking about a particular use case in the context in which you want to deploy the AI, you’re going to miss things.”
EXPLORE: How are health IT leaders achieving digital transformation success?
Lee acknowledged that conversations about AI “touch a nerve in people.”
“There is something that is beyond technical or scientific or ethical or legal about this,” he said. “It’s a very emotional thing.”
Because of this, Lee said, it’s important for people to get hands-on understanding about AI, to learn about it firsthand and then work with the rest of the healthcare community to decide whether such solutions are appropriate.
Moore added that healthcare organizations should have their own teams that understand AI rather than rely solely on vendor knowledge and products.
Keep this page bookmarked for our ongoing coverage of HIMSS23. Follow us on Twitter at @HealthTechMag and join the conversation at #HIMSS23.
