Effective medication management requires a trustworthy intelligence layer
The role of the LLM should be to interpret, surface, and support the underlying clinical content—not change or override it. Some applications of AI are superior choices for clinical solutions—especially in medication dosing when margins of error are small and patient safety is a priority.
Deterministic AI is rules-based and operates on fixed logic. This contrasts with probabilistic or generative AI which works from predictions and likelihood, creating a dynamic where—if not carefully validated—outputs can vary even with identical prompts or even hallucinate, which may impact patient safety standards.
In medication management, AI solutions need this rules-based logic and to be built on content generated and guided by clinical expertise. A 2025 study found that a generic LLM solution scored only 54% on basic drug interaction checks. A 2024 study from the National Institutes of Health linked adverse drug events to 5% to 28% of acute geriatric medical admissions—some of the most severe consequences of inappropriate medication use for older adults. Globally, the World Health Organization attributes more than half of all medical harm to medications.
Medication management cannot tolerate variability—making deterministic standards critical for trustworthy clinical AI. Strong governance at the final stage of the workflow is essential to preserve the integrity of the original expert-driven content.
DIY medication data collides with clinician needs
Many organizations are experimenting with internally assembled medication management workflows using commercially available probabilistic tools. This approach fails to infuse clinician insights and expert content into the entire medication workflow—and often lacks the consistency required to help inform high-risk decisions.
This is a problem because medication information is connected to highly-engaged clinical content. Over the last five years, clinicians using UpToDate® sought dosing guidance in 32% of safety-related protocols. This comes second only to infection prevention searches, which were 40% of all patient safety-related lookups.
High levels of clinician engagement underscore how central medication decisions are to daily clinical workflows—and why gaps in intelligence layers introduce outsized risk.
Medication management workflows are becoming more complex
The intersection of AI and medication decision-making is becoming higher risk as drug development expands and high-alert medications are increasingly in play.
Between 2010 and 2019, the number of new drugs approved for sale by the US Food and Drug Administration increased by 60%. From 2023 to 2025, there was a continual flow of drug approvals with 151 novel drug approvals and 61 new biological treatments.
The risk implications are high when it comes to medication management. If AI hallucinates or guesses, the smallest dosing math error can impact patient safety. The most frequently searched high-alert drug topics in UpToDate from 2021-2025 are:
- DOAC dosing – 1.61 M
- Heparin & LMWH – 1.43 M
- Warfarin/VKAs – 773,000
- Vancomycin - 673,000
- Digoxin toxicity/dosing – 567,000
- Aminoglycosides – 430,000
In these scenarios, AI supported by governance and built on expert-generated clinical content is critical to support reliable care decision-making. Grounding AI governance in these standards can also reduce the need for high-risk trial-and-error with solutions not purpose-built for healthcare enterprise needs.
