Ungoverned AI can put clinical judgment at risk
Modern clinicians are increasingly exposed to ungoverned AI that can shape decision-making without consistent standards for validation, transparency, or accountability. When tool governance is minimal or non-existent, clinicians turn to unsanctioned tools not purpose-built for healthcare, compounding risk by introducing outputs into clinical decisions that are disconnected from evidence, local standards, and governance. These risks are not just technical; they can directly influence how clinicians make decisions, what they trust, and how much reasoning they retain over time.
The risk goes beyond unauthorized software. Health systems often have their own walled gardens of information, like approved local formularies that unsanctioned tools can’t easily access. Additionally, ungoverned AI tools may monetize user data through advertising or lack transparency in how outputs are generated, introducing risks that are not always visible to clinicians or patients, yet may still shape decisions at the point of care.
The rising risk of clinical deskilling from AI
According to the Future Ready Healthcare Report, almost three-quarters of physicians are concerned that clinical deskilling will be one of the most significant risks of over-reliance on AI tools. AI deskilling can erode critical reasoning, specialized knowledge, and practical abilities by taking over core tasks from humans and creating undeserved trust in generative AI outputs—an issue that compounds over time.
This could look like:
- Relying on AI-generated differential diagnoses without independent validation
- Reduced exposure to complex clinical reasoning during training
- Over-trusting well-written but incorrect outputs
We’ve already seen this with physicians newer to practice who no longer know how to write effective paper notes when a system goes down versus those of us who do, or those who may over-rely on AI tools in medical school. If left unaddressed, this risk will only increase as decision-support models automate more processes in diagnostics and treatment planning.
To counter this concern, AI upskilling empowers clinicians to engage with AI in ways that enhance their clinical skills and judgment. Effective upskilling must be intentional, requiring enterprise-level training that supports AI literacy and strengthens clinical expertise. Intentional AI literacy, combined with governance built on experts-in-the-loop, will separate future-ready organizations from those that struggle to achieve productive applications of AI.
Future-readiness starts with operationalized governance
To meet this moment, health leaders need to infuse governance and expertise into their operating model, starting with clinicians.
Integrating clinicians into solution selection is critical to effective governance programs. Healthcare leadership should get input on the best ways to develop AI literacy and collaborate to identify trusted, expert-driven tools for evidence-based shared decision-making. Clinicians can get support from leadership, starting with:
- AI literacy and governance training through peer accountability and shared learning.
- Sanctioned tools that employ expert-in-the-loop models by bringing expert-reviewed evidence into shared conversations with patients.
- Team- and organization-wide feedback loops and monitoring that can address their needs.
These elements create a self-reinforcing governance system that reduces the burden on clinicians to individually manage AI risk while elevating their role as confident orchestrators of the AI-enabled care team.
AI translates to success at the point of care
Ultimately, as AI spreads across clinical workflows, the clinical context and sources are foundational for decision-making and clinician-patient partnerships.
I recently had a patient whose skin biopsy came back with a finding that wasn’t something I had previously encountered. UpToDate® Expert AI highlights that this diagnosis can be difficult for both clinicians and patients. Reviewing the evidence together, with clear visibility into its clinical authorship, helped ground the conversation in trusted guidance. This is how governed AI can help support more informed, transparent decision-making, and it allowed me to provide a level of reassurance to the patient that made her confident in the care plan.
AI will only improve care outcomes if clinicians remain actively engaged, not just as users of technology, but as critical evaluators of it. Organizations that operationalize governance by investing in AI literacy and upskilling will not only reduce risk, but will also define how clinical expertise evolves in an AI-driven care model.
Explore more on clinician enablement in our latest report, “UpToDate Point of Care Report: Patient safety in the AI age.”
