CHAI releases checklist for responsible AI implementation

The process of infusing artificial intelligence into the digital health ecosystem is like nothing that the industry has ever encountered before. Even during the frenetic EHR boom in the early 2010s, organizations could rely on a certain degree of guidance from developers and regulators, and a shared framework of standards, that at least defined the end goals of AI implementation, if not a single best way to get there. 

The AI boom stands in stark contrast with few rules and, arguably, even less cohesive understanding of the ultimate end point. Combined with the false sense of urgency produced by the peak of the hype cycle – and the very real financial and clinical challenges that are forcing leaders to scramble for anything that can relieve the pressure – organizations might easily find themselves at the center of a perfect storm of technology missteps.  

In an effort to help prevent digital disasters, dozens of entities have stepped up to offer their perspectives on what ought to be done with AI and how healthcare providers should go about doing it. Unfortunately, with more than 200 different sets of AI guidelines floating around in the academic ether, the sheer number of documents and lack of consensus among these good-faith efforts may be more confusing than illuminating. 

The Coalition for Health AI (CHAI) has several entries in that list, but is trying to bring stakeholders together to reduce the noise and offer more effective “cheat sheets” for implementing AI in a trustworthy, responsible, and effective manner. 

Their latest effort is the CHAI Assurance Standards Guide, which offers detailed insight into the emerging areas of AI governance throughout the technical lifecycle. 

Building on its previously published framework of core principles for trustworthy AI, the new guide provides step-by-step tips for putting these principles into action during the six major phases of a real-world implementation.  

Defining the problem and planning solutions

The process starts with the most basic question: what problem needs to be solved? Organizations should clearly define the issue and be able to explain why a specific AI solution is the best answer.  

During this phase, leaders should conduct a risk-benefit analysis, evaluate impacts to users and workflows, and consider potential issues such as the potential for bias and the need for transparency, as well as potential effects on safety, patient experiences, privacy and security, and compliance. 

Designing the AI system with safety and fairness in mind

Risks to patient safety must be top of mind when entering the design phase, especially in tools that touch the clinical environment. CHAI suggests focusing on safety, fairness, and bias by creating extensive documentation throughout the process and giving users a clear path for providing feedback when something doesn’t seem right. Risk assessments should be conducted throughout the design step, and organizations should develop procedures to monitor for adverse events and manage any ethical, compliance, or other legal challenges.   

When possible, AI systems should be designed to keep humans in the loop as the technology develops. This can help ensure that the model’s decision-making is explicable to a human user and the results remain useful in real-world settings. 

Engineering the AI solution

When engineering the solution, organizations will need to focus on using the best possible training data. Training datasets should be representative and inclusive while closely matching the target population, and should offer insights into any proxies, composite scores, or other statistical methods that might influence its results.  

Thorough documentation of data provenance can help foster accuracy and transparency, while strong data management policies can ensure that all privacy and security guidelines are being rigorously applied to protect patients throughout the data lifecycle. 

Assessing and evaluating the model in situ

Much can change between the initial planning phase and the implementation process, so leaders will need to reassess how the model is actually likely to perform once the bulk of development has been completed.  This includes taking another look at how the model actually solves the initial program, reassessing impacts on workflows, reinforcing methods to detect and address bias or harm, and establishing methods to validate and monitor ongoing results. 

During this stage, leaders should also ensure that users understand their cybersecurity roles and responsibilities when using AI-driven tools to avoid privacy and security concerns. 

Piloting the model in a real-world situation

When launching a pilot, organizations will need to carefully monitor how users interact with the model, including what happens when humans disagree with the AI’s output. Regular reassessment – or better yet, continuous monitoring – of the algorithm in practice can help to avoid risks related to quirks in the training data or incorrect use by human users. Organizations should pay close attention to emerging patterns in clinical care and outcomes, and should be prepared to move quickly with a clearly structured, transparent, and repeatable decision-making process should evidence of harm emerge.  

Deploying a model at scale

 Over time, AI models can experience “drift” in their results as data is reincorporated into the algorithm. Organizations will need to carefully watch for this phenomenon and develop a strategy to address the situation accordingly. They will also need to consider how patients and other affected parties will be informed of the use of AI as part of their care, and what to do if a patient wishes to opt out of participation. 

Throughout the ongoing deployment, organizations must continue to prioritize monitoring for safety and efficacy, and conduct regular impact analyses with standardized metrics for high-value outcomes. 

The full guide includes a much more detailed checklist for ensuring that each step of the AI implementation process is completed as smoothly and successfully as possible.  

Staying focused on key areas of transparency, governance, safety, and security will give organizations a head start on making the most of this new generation of tools – and assist with generating evidence of tried-and-tested best practices for the next wave of AI technologies that will support better long-term outcomes for technologists, clinicians, and patients alike. 

Jennifer Bresnick is a journalist and freelance content creator with a decade of experience in the health IT industry.  Her work has focused on leveraging innovative technology tools to create value, improve health equity, and achieve the promises of the learning health system.  She can be reached at jennifer@inklesscreative.com.