“AI assurance” and trust goes beyond the care team
As AI becomes embedded in clinical workflows, evaluation criteria must go beyond performance claims. Outsell predicts that AI governance will soon be treated like cybersecurity or regulatory compliance: a core purchasing requirement rather than a secondary consideration.
For leaders considering patient safety and quality requirements in AI-powered clinical decision support, some questions to ask vendors:
- How is the system grounded? Do outputs come from trusted, evidence-based clinical content or general model knowledge that may be incomplete or outdated?
- How is it evaluated? Is the system assessed against real clinical standards such as clinical intent, knowledge integrity, and potential risk, or just generic benchmarks?
- Who defines “correct”? Are physician experts involved in setting evaluation criteria and reviewing performance?
- How is it monitored over time? What processes are in place for continuous evaluation, governance, and improvement?
Trust and governance are emerging as a new category of enterprise value, something Outsell describes as the rise of “AI assurance.”
As clinicians become confident in their AI tools, they’ll incorporate it into their decision-making, using it to think more deeply, not less. These benefits can extend beyond clinicians, helping support:
- Standardization of care across sites and teams
- Alignment with evidence-based practices
- Greater confidence in system-wide deployment
A trusted clinical AI framework is the new differentiator
Outsell emphasized that the clinical AI evaluation framework used to develop UpToDate® Expert AI operationalizes trust across three key areas relevant for real-world practice:
- Clinical intent: Physician-authored rubrics spanning 25 specialties define quality. In testing, UpToDate Expert AI met 99.9% of assessed clinical criteria.
- Knowledge integrity: Answers are derived directly from curated clinical sources rather than unverified model knowledge.
- Potential risks: How does the system behave under stress or ambiguity? Human experts define and evaluate potential harm.