The healthcare industry has never lacked information. From electronic health records (EHRs) to imaging, genomics, and patient-generated data, the healthcare system is saturated with inputs. Many see generative AI (GenAI) as a solution to synthesize this information overload and data complexity.
But synthesis alone isn’t enough. According to a survey by IBM, nearly half of executives believe their employees lack the necessary skill to implement AI technology at scale, and the Center for Governance of AI estimates 6.1 million American workers are both exposed to AI in their jobs and ill-equipped to adapt to it. Nonetheless, interest in and pressure to implement AI-enabled solutions keeps growing.
Thus, healthcare leadership involved in procurement, compliance, and healthcare AI governance are now prioritizing AI literacy, including improving basic user skills and knowledge of AI-enabled healthcare solutions. At the same time, this shift is signaling something deeper to digital health tech developers: That the success of AI in healthcare will depend not just on model performance, but on how effectively end users understand, interpret, and apply its outputs. This presents digital health tech developers with both an opportunity and mandate to provide advanced solutions that not only deliver reliable, secure AI, but also embed AI literacy directly into the user experience.
Understanding AI risk and its impact on patient and health consumer trust and outcomes
Healthcare AI doesn’t just deliver information; it shapes decisions and helps influence actions.
“Decisions can be altered by presenting information,” explains Dr. Peter Bonis, Chief Medical Officer at Wolters Kluwer Health. “About 30% of the time, healthcare professionals will change their decision if they're presented with information at or near the point of care. So, if that information in some way is faulty, it can lead to at least suboptimal care, if not impair patient safety as well.”
When AI-generated outputs are misunderstood, misapplied, or blindly trusted, the consequences are amplified. This makes GenAI tools difficult for healthcare procurement teams to evaluate, Bonis says, because they are “evolving as they are being integrated into the workforce,” and it is often uncertain how they will ultimately fare in terms of clinical reliability.
Watch Dr. Peter Bonis discuss how – when AI shapes decisions – trust is on the line.