Clinicians and patients are calling for greater oversight in healthcare AI use

AI adoption is accelerating, but trust is not keeping pace. Clinicians and patients are already actively defining boundaries around how AI should be used in the healthcare setting, with those limits largely centered on preserving human oversight, critical thinking skills, and decision-making abilities.

Both groups are advocating for robust governance of AI tools to prevent potential problems with how AI will be used in practice.

For clinicians, the concerns largely center on retaining control and autonomy over their ability to make trusted, independent clinical decisions. For example, three-quarters (74%) anticipate that clinical “deskilling” will be one of the greatest risks of overreliance on AI tools, especially as decision support models offer to automate more of the process around diagnostics and treatment planning. Just over two-thirds (68%) of patients share these concerns.

AI productivity tools have unintentionally created an additional workflow step for doctors and nurses: the validation of AI-generated clinical content.

The risks of deskilling are compounded by ongoing worries about AI hallucinations, which may infuse inaccurate or irrelevant information into the decision-making cycle. Once again, three-quarters of clinicians (74%) cited hallucinations as a major concern affecting their ability to practice appropriately. And while 73% said they are somewhat or very confident in being able to determine whether an answer is clinically valid without consulting an outside source, that still leaves a quarter of clinicians who are simply not sure if they can identify incorrect output without cross-verification.

Most (77%) clinicians will take the extra step to verify AI outputs, however, largely by confirming with trusted databases like UpToDate and PubMed or clicking on citations and source links. A similar number (78%) of patients expect them to do so, illustrating alignment in the need to double-check AI answers.

How concerned are clinicians and patients about AI-driven clinical deskilling?

Bar chart showing percentage of clinicians and patients concerned about “deskilling” due to over-reliance on AI, with all groups reporting high concern levels.

  • Graphic description

    This bar chart displays the percentage of different groups who are concerned about “deskilling” as a result of over-reliance on artificial intelligence. The chart includes four categories: total clinicians, doctors, nurses, and patients. Concern is highest among doctors at 77%, followed by total clinicians at 74%, nurses at 70%, and patients at 68%. All groups show a majority level of concern, with percentages ranging from 68% to 77%. The chart highlights that concerns about reduced skills due to reliance on AI are widespread across both healthcare professionals and patients, with clinicians reporting slightly higher levels of concern than patients overall.

Transparency and validated sources are key to building trust in healthcare AI

The majority of clinicians (59% of nurses and 72% of doctors) want to see those citations and sources presented clearly within the workflow to make it easier to verify information. In addition, about half of clinicians say they want AI to be required to show detailed reasoning behind its responses.

This expectation is shaping how both groups think about AI transparency and verification from the ground up. More than 90% of clinicians (and 89% of patients) feel it is important for human experts to be in the loop to validate the sources behind AI-generated content to reduce the likelihood of errors creeping into the system to begin with.

Seeing does not equal believing: Building trust in healthcare AI depends on expert-in-the-loop

Roughly 90% of both clinicians and patients felt that it was very important or somewhat important to have the sources and AI systems used to generate clinical content be validated by a human expert-in-the-loop. This underscores the trust gap that still exists when it comes to using AI for critical decision-making tasks.

Nurses

Doctors

Patients

AI in Healthcare raises growing concerns around privacy, bias, and accountability

Meanwhile, patients surfaced additional concerns about data privacy and security (74%), as well as AI-driven biases (72%) and lack of clarity around accountability if AI contributes to a poor outcome in their care (75%). Clear labeling of AI use within the care process would help, said 64% of respondents.

Interestingly, clinicians are more in favor of strong penalties for misuse of AI or data breaches than their patients. Half (49%) of doctors want to see more significant consequences for using AI incorrectly or failing to protect personal data, while just 29% of patients said the same.

Despite these expectations, governance remains largely invisible. Most clinicians still aren’t aware of published policies within their organizations. Just 27% of doctors and nurses said they knew about how their workplaces are addressing governance issues – up from 21% in 2025 – which leaves a major gap in governance education for leaders to fill.

Clear guardrails are essential to building confidence in healthcare AI output

Among those who were aware of organizational policies, most (63%) understood how established privacy regulations like HIPAA apply to AI use, and 51% were aware of policies explicitly listing approved and unapproved tools. Significantly fewer (35%) knew about guidelines for validating the accuracy and reliability of output, and just 22% said their organization had policies for delineating the responsibilities of clinicians and AI tools.

Overall, nurses were more aware of the details of the policies in place than doctors, highlighting an opportunity for additional conversations with doctors about AI guardrails. As AI becomes more embedded, closing that gap will be critical. Without clear, accessible guardrails, even well-designed tools risk eroding the trust they are meant to build.

Awareness of AI policies in healthcare organizations

Bar charts comparing 2026 and 2025 responses from clinicians, doctors, and nurses, showing most respondents say “No” with smaller increases in “Yes” responses in 2026.

  • Graphic description

    This graphic presents three sets of horizontal bar charts comparing responses from total clinicians, doctors, and nurses in 2026 and 2025 across three categories: “Yes,” “No,” and “Don’t know.” For total clinicians, 27% responded “Yes” in 2026 compared to 21% in 2025, while “No” responses decreased from 54% in 2025 to 44% in 2026. “Don’t know” responses increased slightly from 25% to 29%. Among doctors, 21% answered “Yes” in 2026 versus 14% in 2025, while “No” responses declined from 55% to 47%. “Don’t know” responses remained similar at 32% in 2026 and 31% in 2025. For nurses, 36% reported “Yes” in 2026 compared to 26% in 2025, while “No” responses decreased from 54% to 40%. “Don’t know” responses rose from 20% to 24%. Overall, the data shows an increase in “Yes” responses and a decrease in “No” responses across all groups from 2025 to 2026, indicating growing acceptance or agreement over time.

Healthcare professionals in a clinical setting collaborating while reviewing information on a tablet, with one individual in a white coat providing guidance to two colleagues in scrubs.

Explore The Future Ready Healthcare Survey Report

Take the pulse of where AI is working in the clinical setting and where there is opportunity to do more as we better understand the unique perceptions of doctors, nurses, and patients around agentic and generative AI.

Report Home
Report Home
Key Survey Findings
Key Survey Findings
Looking Ahead
Looking Ahead
Abstract image of flowing, multicolored light waves in blue, purple, and red tones against a dark background, suggesting motion and digital data movement.
Back To Top