Trust in Clinical AI Starts with Governance

Making clinical AI reasoning useful

In my field, transplant infectious diseases, knowledge evolves quickly, the medicines are complex, and the consequences of error can be profound—especially if clinical AI tools derive an answer from a single article. Expertise means bringing the body of knowledge to the bedside, from basic science to clinical studies, while understanding what each can establish.

Consider a lung transplant recipient with persistent cytomegalovirus (CMV) despite antiviral treatment. Interpreting the viral trajectory requires knowing the treatment duration, drug exposure, and patient’s immune state. Choosing another antiviral brings resistance, toxicity, and interactions with immunosuppressants into the same decision.

Making that decision requires bringing viral biology and pharmacology together with the study populations, outcomes, and wider clinical experience. The maribavir trial, for example, excluded CMV retinitis and central nervous system disease; those limits matter when applying its findings. Simply finding a closely related trial and summarizing it does not do that job.

Validation also reaches beyond the answer to its effect on the clinician. In a pilot experiment, six radiologists read chest X-rays with and without simulated AI feedback. On cases where the simulated AI incorrectly signaled no abnormality, their false-negative rate rose from 2.7% without AI to 20.7%–33%, depending on the presentation. The study was small and experimental; it illustrates why the interaction merits evaluation alongside the output.

Does an answer help a clinician notice missing information, reconsider an assumption, or explain a choice to a patient? Those questions connect validation to the work of care.

Governance we can examine

For healthcare leaders, governance becomes tangible when a developer can show the work behind a response. A demonstration gains depth when the case changes: introduce a resistance result, a new interacting medicine, or disease at a site excluded from the trial. Does the answer explain why that detail changes the interpretation?

A few questions help make those standards visible:

• Who is clinically accountable? Who sets standards, reviews performance, and investigates problems?
• How is knowledge maintained? How are experts involved in the creation of clinical AI tools and how do they select evidence, resolve conflicting findings, and update guidance?
• Is the clinical reasoning explicit? Can a clinician see the relevant patient factors, alternatives, uncertainty, and supporting evidence?
• What does validation show? How are omissions, unsupported inferences, and unsafe recommendations identified? What do practicing clinicians find useful?
• How does the system improve? How are concerns investigated, corrections tested, and performance reassessed as models and evidence change?

Documented findings give these conversations substance and help build clinician confidence. They also let a smaller hospital examine a system’s clinical foundations without assembling an AI research team of its own.

This work benefits from collaboration among developers, practicing clinicians, and academic researchers. Each sees different failure points. Connecting their perspectives helps turn a bedside concern into a testable question and a practical correction. It is a familiar medical habit: sharing what we learn so the next person is less likely to meet the same problem.

Good clinical practice that fits the work

Clinical governance works best when it fits the work. In a Wolters Kluwer survey, 57% of responding providers and administrators had encountered or used unauthorized AI tools in their organizations. About 27% of providers were somewhat or not aware of AI policies in their organization.

Nearly half of those using unsanctioned tools cited faster workflows; one in three cited an unavailable approved alternative or missing functionality. These findings help explain why adoption can outrun oversight. They also point to a practical response: approved tools that fit the work, with clear guidance on intended use, further review, and what patient information can be entered.

Data handling belongs in that conversation. IBM’s 2026 Cost of a Data Breach Report put the average healthcare breach at about $6.6 million, covering healthcare breaches overall rather than AI alone. Knowing where information goes, who can access it, and how long it is retained makes tool selection and everyday use more informed.

Drug manufacturing offers a useful analogy. Good manufacturing practice builds quality into the process because testing finished products alone cannot establish it. Clinical AI invites a similar view: knowledge selection, clinical validation, data handling, and change control all contribute to the reliability of the system. The answer on screen is one part of a much larger picture.

Trust in everyday clinical care

Governance becomes visible in small details: a source beside a recommendation, an explanation of uncertainty, a way to explore alternatives or report a concern. Good design lets clinicians move from a quick answer to a closer examination without losing the thread. Ease then serves understanding.

As AI appears in electronic health records, documentation tools, and patient portals, evaluation follows the task. A treatment recommendation raises different questions from a draft note. Ongoing review keeps clinical standards connected to changing models, evidence, and use.

For me, building UpToDate Expert AI means making this clinical discipline part of the system itself: answers clinicians can examine, reasoning they can follow, and confidence grounded in evidence.

Explore more in our latest report, “UpToDate Point of Care Report: Patient safety in the AI age.”