Centers for Disease Control and Prevention estimated that about 225,000 C. difficile infections and 12,800 deaths occur each year in the United States. An estimated $4.8 billion is spent annually to manage these infections.
Leveraging the power of artificial intelligence, the data science team at Wolters Kluwer developed a risk assessment algorithm to take on the challenge and identify hospitalized patients who are at high risk for developing C. difficile infections earlier. Scientific evidence has identified associations of patient-specific factors, such as age, organ dysfunction, antibiotics use, gastrointestinal medicine use, etc., to develop C. difficile infections. Because some of the modifiable risk factors are related to medication use, Wolters Kluwer deployed the risk assessment model in 2020 in Sentri7 to provide actionable alerts and guidance to pharmacists who are best positioned to make an impact.
During the validation phase, risk assessment model’s performance was analyzed with the incidence of infection. The data science team found that 75% of patients who developed C. difficile infections scored in the 80th percentile or above, and a mean prediction time of 5 days prior to the onset of infection. The risk assessment model was paired with Sentri7 alerts to identify patients who most likely may benefit from pharmacist intervention to remove or modify high-risk medicines as a risk mitigation strategy. Examples of such alerts include proton pump inhibitor therapy, fluroquinolones and clindamycin use, and prolonged antimicrobial use.
Figure 1. C. diff Risk Score Prediction Validation
Assessing clinical usefulness of C. diff risk scores
Following validation, Carilion Clinic partnered with Wolters Kluwer in a pilot study to assess the usefulness of the risk assessment model and the accuracy of Sentri7 alerts in a clinical setting.
“I don't think there is a health system that has fully figured out CDI,” said Nathan Everson, PharmD, an Infectious Disease Clinical Pharmacist at Roanoke, Va.-based integrated health system Carilion Clinic. “I think for every health system, it is on their hit list for their infection control program. The problem is multi-factorial. Lots of things can affect a patient’s normal flora and predispose them to a CDI. It is something we constantly struggle with.”
Carilion’s antimicrobial stewardship pharmacists used the alerts over three months to identify high-risk patients and intervened by discussing the patients’ medicine regimens with their respective prescribers. High-risk medicines were discontinued where appropriate. During the pilot period, in the highest risk group, 83% fewer C. diff infections were observed when a pharmacist intervened to remove or modify a patient’s medicines. The most common interventions made by the pharmacists at Carilion Clinic were to discontinue gastrointestinal medication and antibiotics.
After using the CDI risk scores for three months, the antimicrobial stewardship leaders updated their Sentri7 dashboards to prominently display the scores so that pharmacists can see the scores for their assigned patients in one place. Sentri7’s real-time prioritization of their work and interventions further sped up removing known risk factors for C. difficile infections.
Carilion Clinic pharmacy team quickly saw its value. “As a multi-hospital system, we face a unique set of challenges in managing patient safety, regulatory compliance and costs, and reducing C. diff infections remains a focus of ours,” said Caitlin Meanor, Infectious Diseases Pharmacist at Carilion. “With the CDI Risk Score embedded in Sentri7, we can harness all of the clinical data associated with patient care and allow our clinical pharmacists to triage management based on risk factors and acuity.”
Key learnings from this project
Carilion Clinic and Wolters Kluwer are pleased with the results of using this C. difficile risk assessment tool to enhance patient care. A few key takeaways that contributed to the successes of the pilot include:
1. Incorporate clinician input early
An essential step of developing a machine learning algorithm is known as feature engineering. This is a process where domain expertise is needed to discern the relevant features of a model. Electronic medical records contain a vast volume of data. However, it’s important to remember the adage ‘correlation does not imply causation.’ Leveraging the clinical expertise at UpToDate, Lexicomp, and Sentri7, the Wolters Kluwer data science team integrated its deep domain expertise in infectious diseases, gastroenterology, epidemiology, and infection prevention into the feature engineering process. The algorithm was analyzed to ensure that risk factors made sense clinically.
2. Actionable alerts empower clinicians in risk mitigation
Even a highly accurate risk assessment model is of little use if it does not improve patient outcomes. The Wolters Kluwer team worked closely with bedside clinicians to design and test alerts to identify modifiable risk factors in high-risk patients, incorporating the latest evidence on C. difficile infection. The result of this assessment was the creation of a bundle of actionable alerts with concise recommendations on mitigating those risks.
3. Integrate AI tools into clinician workflow at the point of care
Next, the AI-driven risk scores and alerts were deployed in the Sentri7 dashboard, which is already part of the pharmacist’s daily workflow. This integration ensures the easiest path to adoption and improves the clinician response time to intervene in patients with high risks.
4. Transparency and communications are crucial to adoption
‘Alert fatigue’ is a well-known phenomenon experienced by clinicians. The data science and clinical teams were intentional in sharing the risk factors that were used to build the model and provided model performance results frequently and clearly. Furthermore, a pharmacist-prescriber conversation often occurs prior to an actual intervention being performed. Wolters Kluwer provided detailed communication aids on how the model was developed to facilitate these conversations, which resulted in high prescriber acceptance of pharmacist recommendations.
Wolters Kluwer saw an opportunity to leverage AI, identify at-risk patients sooner, and help clinicians proactively address modifiable risk factors and prevent C. diff infection using known, evidence-based prevention strategies. With AI-powered surveillance software, hospitals and health systems are one step closer to staying ahead of hospital-acquired infections like C. diff, which is crucial to reduce mortality risk, length of stay, and the potential for financial penalties.