DynaMed CMO’s three steps for safer clinical reasoning with AI: Ask, audit, apply

For the everyday user, the real magic of LLMs has nothing to do with transformer architecture, billions of parameters, or the massive datasets underpinning the foundation models.

It’s the ability to ask a question in simple, everyday language, with all the usual “likes,” “umms,” and “kindas” included, and get an accurate answer that understands what you really meant rather than what you actually said.  

General-purpose LLMs are usually quite good at interpreting these imprecise queries, at least when it comes to identifying that restaurant you heard about on Instagram or figuring out which actor was in that movie with the guy from the thing.

But when it comes to clinical-grade decision support tools, in situations where getting to the right answer quickly could determine a patient’s outcome, it isn’t good enough to just throw a vague query at the wall and see what sticks.

Precision matters in clinical decision-making. And precision begins with knowing how to formulate the right query, analyze the AI output, and appropriately use the information to make the best possible decisions about the patient’s needs.

Clinicians need a specific skillset to make this happen, says Roy Ziegelstein, MD, MACP, a practicing cardiologist, professor of medicine at Johns Hopkins University School of Medicine, and editor in chief and CMO of DynaMed.  

But unfortunately, many clinical AI users aren’t getting the training and education they need to use their AI tools in a safe and effective manner during the clinical decision-making process.

“It’s assumed that if you can talk to Siri, or if you can put something in Google, that you know how to formulate the right question to put into AI for clinical decision support. That is absolutely incorrect,” he told Digital Health Insights. 

“We have a lot of practicing clinicians using AI, whether it’s for clinical decision support or for other purposes, who actually have never been trained on using AI. At the same time, we know that automation bias is a problem, where we tend to accept whatever answer is provided by the tool, even if it’s not correct.  It’s potentially a recipe for disaster, unless we improve the way we teach AI users to interpret and apply these insights.”

A three-step framework for an AI-enabled “reasoning loop”

Technical training on specific tools is important, but so is having a cognitive model for how to integrate AI-generated information into a patient-centered, person-powered decision-making process.

To assist clinicians with structuring their thinking around AI CDS, Ziegelstein and colleagues developed the Ask, Audit, Apply (AAA) framework: a three-step “reasoning loop” that defines the “behavioral obligations” involved in using AI for clinical decision support.

“AAA integrates principles from evidence-based medicine, diagnostic reasoning, automation bias research, and emerging AI governance literature into a workflow-embedded process that preserves human judgment and accountability,” explains an article detailing the approach, published in the Journal of General Internal Medicine.

“Although conceptual and not yet empirically validated, AAA articulates decision-level reasoning practices that may support education, supervision, and future research on safe AI integration across a range of CDS tools as these systems increasingly participate in clinical care.”

Ask: deliberately formulating actionable clinical questions

Whether using natural language or more structured query techniques, giving the CDS tool accurate, relevant information is crucial for returning the most meaningful result.  Doing so requires a firm grounding in clinical reasoning and the underlying understanding of when and where AI can appropriately supplement the process.

“Every Friday morning, I teach residents and students at Johns Hopkins how to think about patients in terms of clinical reasoning: how to obtain information from the patient history, the physical examination, and diagnostic tests, and then formulate a differential diagnosis and pare down that differential into an actual leading diagnosis,” said Ziegelstein. 

“AI can assist in that process, but it doesn’t replace that process.  You have to know what information to put into the AI in order to yield information that’s useful clinically.”

Posing queries that clearly specify a defined clinical question, supported by relevant context, is an “accountable act that shapes the downstream validity of AI output,” the team wrote.

Audit: Asking the right questions about the right(?) answers

“Trust but verify” has always been an important mantra for technology users, never more so than in the age of AI.

“When AI provides an answer, you have to be able to test whether or not it holds water,” said Ziegelstein. “We do that currently when we get a search result that doesn’t quite make sense, right? Maybe I put in someone’s name, and it returns someone who seems like a totally different person. I’m going to investigate that and maybe revise my query or add more details until I get something that is more aligned with what I need. The same thing has to be done clinically. That’s the audit part.”

When auditing an AI response, users should first determine if the answer is informational only or whether it includes a recommendation for action. If there’s a recommendation, clinicians should evaluate “the evidentiary basis, recommendation strength, and transparency of AI-generated recommendations,” said the team, and “look for a statement about the methods used to obtain responses.”

Users should make an effort to independently evaluate or verify the recommendation with evidence-based resources and check if it aligns with clinical best practices and guidelines.

“Critically, Audit requires clinicians to actively discriminate among levels of evidence and to recognize when AI output extends beyond validated indications or relevant patient populations,” writes the team. “Audit is therefore not about distrusting AI by default, but about calibrating trust appropriately and preserving epistemic responsibility.”

Apply: Integrating the advice and owning the outcome

“AI does not make decisions. Clinicians do,” the paper firmly states.

That means that clinicians are responsible for defining the role that AI-enabled input plays in their overall decision-making arc – including ensuring that AI insights aren’t the only thing influencing their care decisions.

“I hope it stands to reason that every piece of information provided by the AI clinical decision support tool has to be tailored to the individual in front of you and integrated into the whole spectrum of information you have about that person, their values, and their expectations,” Ziegelstein said. 

“When we look at the newest journal articles, we don’t just say, ‘well, this is the thing we’re all doing now for everyone with this condition.’ We look at the potential benefits, the potential harms, and how those match up with what’s meaningful to the individual. It’s the same with AI answers.  The responsibility is on the clinician to be intentional about how we use this data as part of a holistic approach to patient care.”

Expecting active participation in applying AI-generated insights

The AAA approach offers a way for clinicians to approach the use of AI CDS in patient care while preserving the central role of clinical judgment. Ziegelstein and his coauthors stress the importance of coaching clinical users in these techniques as early as possible in their training to provide a structured, repeatable methodology for continuously learning and refining their AI skills.

“Clinicians need to view AI as a facilitator that can provide high-value detail and perspective. But it’s to be used alongside those fundamental clinical skills,” concluded Ziegelstein.

“I’m very hopeful that by using AAA alongside AI tools to access richer information more rapidly at the point of care, clinicians can spend more time getting to know their patients as individuals and work together to produce the best possible outcomes.”

Jennifer Bresnick is a journalist and freelance content creator with a decade of experience in the health IT industry.  Her work has focused on leveraging innovative technology tools to create value, improve health equity, and achieve the promises of the learning health system.  She can be reached at [email protected].