Shadow AI: Transforming risk into responsible innovation

The ubiquity of shadow AI has caught the attention of some of the most experienced and influential leaders in our healthcare institutions. To explore the risks, potential, and strategic responses to this emerging technology, Wolters Kluwer Health partnered with Becker’s Healthcare to host the webinar, Shadow AI: How 5 systems are managing risk + empowering responsible innovation.

The discussion addresses some of the most pertinent and prescient topics shaping the use of generative AI (GenAI)—including provider agency, the positioning of sanctioned enterprise-grade AI solutions, as well as actionable options for healthcare leaders who recognize the importance of the current moment.

GenAI is an exciting new technology for both organizations and users. This dynamic propels change and progress from the user level, while also prompting new challenges at the macro level in the US and around the world.

Shadow AI is emerging in multiple forms

Shadow AI use is emerging most prominently across clinical, administrative, and operational workflows. These uses include:

• Chatbots.
• Writing and drafting functionalities.
• Image and visual design, video and audio production.
• Coding tools.
• Education and productivity.

There was a shared challenge of users in the panel’s respective organizations sometimes turning to unsanctioned tools even when internal tools were made available—again highlighting the question of training, awareness, and involvement in decision-making.

Shadow AI is ubiquitous, and guardrails remain challenge

The discussion confirmed much of what emerged from a Wolters Kluwer Health survey of 500 administrators and frontline clinicians.

• Use of Shadow AI is already pervasive, with more than half of respondents encountering unauthorized use of AI tools.
• Clinicians and staff are turning to shadow AI because it improves efficiency.
• Deploying benchmarks and guardrails is still challenging because of hallucinations, reliability, and recursiveness.

There is increasing data that suggests even experts aren’t recognizing when they’re presented with out-of-date or false information. A 2023 NIH study on radiologists found that when AI tools incorrectly flagged no abnormality, false negatives increased from 2.7% to between 20.7% and 33%. Beyond this, the clinical reasoning that undergirds a tool’s decisions is often faulty. These are risks that compound and impact clinicians themselves. Professionals can become increasingly reliant on AI, even as the tools lose effectiveness over time.

Robust governance committees are an important first step, but they will still struggle to address these hurdles.

Governance and leadership can meet the challenges of shadow AI

A murky future of shadow AI calls for governance models and leadership tactics that match the complexities of the technology. These can include:

• Taking full inventory of available AI tools across all departments to surface opportunities to expand, scale or contract before introducing new solutions.
• Informing clinical and non-clinical staff of governance policies.
• Using surveys and recurring meetings to guide and shape innovation at the organizational level.
• Piloting in secure environments to identify hallucinations and perform due diligence under controlled conditions.
• Bringing in the board of directors to help establish expectations around risk and oversight while setting a pace for important projects.

Unpacking a potential future of shadow AI

AI is an incredibly complex challenge, but there are macro-level changes that healthcare institutions can access to catalyze progress in areas where its impact is particularly poignant. One of the most visible is Centers for Medicare & Medicaid Services’ Rural Health Transformation Program, which provides training and technical assistance for the development and adoption of technology-enabled solutions that improve care delivery in rural hospitals.

AI is a work-in-progress in making demonstrable change in areas such as access to care, affordability, delivering consistent quality and safety, and health equity—but a future where the technology reaches its full and most positive potential doesn’t have to be far off.

It has already made a significant and measurable positive impact in areas like clinician workflows, documentation time, and ambient tools. Closing this gap is greatly aided by a strategic, enterprise-level approach as proven guardrails for rapid tool adoption and optimized use.