Explore how an all-electric infrastructure helps improve environmental, operational, regulatory, and energy efficiencies.

The Rise of the All-Electric Hospital

The Rise of the All-Electric Hospital - Digital Health Insights

Home

Digital Health Insights is the healthcare industry’s most trusted source for data, analytics, and executive knowledge.

Digital Health Insights

https://dhix.dhinsights.org/wp-content/uploads/2022/02/dhi-logo-regular-mix.png

This case study details how EvergreenHealth migrated millions of records to their Epic Gallery without disrupting clinical operations.

How a Health System Migrated 32.5M Documents from their Legacy DMS to Epic Gallery in 60 Days

How a Health System Migrated 32.5M Documents from their Legacy DMS to Epic Gallery in 60 Days - Digital Health Insights

Servicing all the devices across a large Midwest health system meant the IT team needed reliable endpoint management that improved efficiency.

3 Reasons Why FCC Behavioral Health Upgraded its Mobile Device Management

3 Reasons Why FCC Behavioral Health Upgraded its Mobile Device Management - Digital Health Insights

A leading regional provider with a complex network used a unified platform to bring together automation, reporting, and real-time monitoring.

How Blanchard Valley Transformed Healthcare IT Operations with a Unified Platform Approach

How Blanchard Valley Transformed Healthcare IT Operations with a Unified Platform Approach - Digital Health Insights

Identity for AI is the trust layer for the agentic era. This guide covers the frameworks, protocols, and governance models to get it right.

The Ultimate Guide to Identity for AI

The Ultimate Guide to Identity for AI - Digital Health Insights

Most enterprises aren't ready to govern AI agents. This brief tells you what AI readiness looks like and how to get there in 90 days.

Are You AI Ready?

Are You AI Ready? - Digital Health Insights

64 staff hours saved monthly. 82% ROI above $500K. Find out what separates high-performing healthcare AI Agent deployments from the rest.

2026 Healthcare AI Agent Benchmark Report

2026 Healthcare AI Agent Benchmark Report - Digital Health Insights

Explore this solution brief to learn how to reduce identity friction and speed time-to-care with passwordless access for clinical staff.

Tap-and-Login with Verified Trust: Modernizing Clinical Identity for Faster, Safer Care

Tap-and-Login with Verified Trust: Modernizing Clinical Identity for Faster, Safer Care - Digital Health Insights

Reduce healthcare security risk and costs with modern identity governance and automated access control.

Strengthening Healthcare Security and Reducing Costs Through Modern IGA

Strengthening Healthcare Security and Reducing Costs Through Modern IGA - Digital Health Insights

Learn how top organizations turn infrastructure into a strategic driver of modern care delivery and resilience beyond the data center.

Most Wired Strategic Intelligence Report: Infrastructure

Most Wired Strategic Intelligence Report: Infrastructure - Digital Health Insights

Gain valuable insights from Rubrik and leading healthcare organizations in this summit on building a robust cyber resilience framework.

Healthcare strikes back: Summit on cyber resilience and recovery

Healthcare strikes back: Summit on cyber resilience and recovery - Digital Health Insights

Hear about top threats, secure cloud migration strategies, and the importance of automation and employee training in this on-demand webinar.

Healthcare cybersecurity: Best practices and approaches (Webinar)

Healthcare cybersecurity: Best practices and approaches (Webinar) - Digital Health Insights

Webinar will review findings from the Cognizant 2023 Healthcare Provider Automation Survey and improving healthcare via automation and AI.

Beyond the Hype: Surprise Insights from the 2023 Healthcare Provider Automation Survey

Beyond the Hype: Surprise Insights from the 2023 Healthcare Provider Automation Survey - Digital Health Insights

Join this event to learn strategies to prepare for and respond to ransomware attacks, and hear insights from C-level healthcare leaders.

Webinar: Ransomware Preparedness for Healthcare: Enhancing Resilience Amid Growing Threats

Webinar: Ransomware Preparedness for Healthcare: Enhancing Resilience Amid Growing Threats - Digital Health Insights

Explore the growing efforts to enhance health equity through emerging technologies (techquity), and the effects on patient populations.

Webinar: How to Make Techquity a Reality

Webinar: How to Make Techquity a Reality - Digital Health Insights

This April 26 webinar will discuss how to prepare for an Epic data center upgrade and the case for high performance compute.

Webinar: Preparing for an EPIC upgrade

Webinar: Preparing for an EPIC upgrade - Digital Health Insights

Digital Transformation in healthcare will require some fundamental changes to cybersecurity in healthcare.

State of Cybersecurity in Healthcare Virtual Summit (On Demand)

State of Cybersecurity in Healthcare Virtual Summit (On Demand) - Digital Health Insights

Webinar on how Scripps Health is improving pop health and maximizing value in direct-to-employer arrangements through digital health.

Webinar: Digital Health Impact on Health System Employer Partnerships and Pop Health

Webinar: Digital Health Impact on Health System Employer Partnerships and Pop Health - Digital Health Insights

Experts from Deloitte and Google Fitbit will discuss the impact of technology advancements on early diagnostics and intervention.

Webinar: A wellness-first world: How early diagnostics and connected technology can shape the future of health

Webinar: A wellness-first world: How early diagnostics and connected technology can shape the future of health - Digital Health Insights

Health care leaders must consider digital inequity when making investments in digital consumer experience.

Webinar: How digital inequity will undermine health care consumer experience

Webinar: How digital inequity will undermine health care consumer experience - Digital Health Insights

The bill would put wearable makers and health app developers under HIPAA-equivalent federal rules for the first time.

What’s next for the Senate bill that puts wearable health data under HIPAA-style rules?

Comment

What’s next for the Senate bill that puts wearable health data under HIPAA-style rules? - Digital Health Insights

https://secure.gravatar.com/avatar/aa42324f12904079e10641bb8ddfc1f70ab0e633b6ef0e65d4f52b31243ff0f4?s=96&d=mm&r=g

Pete Pappas

As providers move from standalone AI scribe tools to a platform approach, the flexibility is supporting specialties across the enterprise.

From tool to platform: The next phase of AI scribe adoption

From tool to platform: The next phase of AI scribe adoption - Digital Health Insights

Dr. Cibotti is a primary care physician with extensive experience in healthcare leadership and innovation. She received her undergraduate degree from Harvard and medical degree at Brown University. After her primary care residency at New York Hospital she returned to Boston and spent 20 years at Massachusetts General Hospital and on the teaching faculty at Harvard Medical School. Subsequently, Dr Cibotti joined Iora Health where she served as the Market Medical Director and opened 2 new primary care practices serving the needs of frail elders. In the past 8 years, she has held multiple roles at Beth Israel Lahey Health including Chief Innovation Officer of Primary Care and Associate Chief Medical Information Officer. Her accomplishments include launching BILH’s virtual primary care program, implementing ambient AI scribes and helping to lead a system wide Epic implementation. In January 2026, Dr. Cibotti joined Heidi Health as the US based Chief Medical Information Officer.

https://secure.gravatar.com/avatar/e702f6f4fe19d44cce4974d7b0b1bdf2356ca9047bd8714bf63292b7618e55ef?s=96&d=mm&r=g

Nancy Cibotti, MD

Hackensack Meridian's Joel Klein, MD, on when enterprise platforms are enough and when a bridge vendor earns its place.

Enterprise AI or point solution? How Hackensack Meridian's CDIO makes the call

Enterprise AI or point solution? How Hackensack Meridian&#8217;s CDIO makes the call

Enterprise AI or point solution? How Hackensack Meridian's CDIO makes the call - Digital Health Insights

https://secure.gravatar.com/avatar/779082e10ef3d2b24bc8e9c0428b01bcb904a659a3ba03554919bbfff12717bc?s=96&d=mm&r=g

Jennifer Bresnick

The agent lied to developers to get malicious code approved and kept the act up until it ran out of tokens.

An AI agent faked identities and doctored its own history in a UK government safety test

An AI agent faked identities and doctored its own history in a UK government safety test - Digital Health Insights

Doctors trusted an AI's patient classifications over their own evidence in two experiments — including one where the treatment did nothing at all.

Research suggests that doctors trust AI over evidence

Research suggests that doctors trust AI over evidence - Digital Health Insights

https://secure.gravatar.com/avatar/7142977f744eb740df152ef54ead95bd4719ca21b0f910df15b86a68d968a31e?s=96&d=mm&r=g

Richard Pallardy

Hackensack Meridian Health has become the first to earn the Joint Commission’s new certification aimed at demonstrating AI governance, safeguards, and use.

Hackensack Meridian earns first Joint Commission AI certification and says every tool gets traced like a hospital survey

Hackensack Meridian earns first Joint Commission AI certification and says every tool gets traced like a hospital survey - Digital Health Insights

Shorter vendor contracts, pooled purchasing through the Ohio High Value Network, and a seat at the advocacy table: Linda Stevenson's plan for doing more with less.

How a rural Ohio health system is absorbing OBBBA's impact

How a rural Ohio health system is absorbing OBBBA&#8217;s impact

How a rural Ohio health system is absorbing OBBBA's impact - Digital Health Insights

https://secure.gravatar.com/avatar/18172c596718523f0c4bbe9ae4122841245197c9a3a03a77140851d04b0dab5f?s=96&d=mm&r=g

Kate Gamble

Abbott, Stryker, Intuitive, Medtronic and other medtech companies disclosed cyberattacks this year. None involved a device — and the pattern should reshape how hospitals assess vendor risk.

Medtech breaches keep hitting corporate IT, not devices — and that's a blind spot in hospital vendor risk

Medtech breaches keep hitting corporate IT, not devices — and that&#8217;s a blind spot in hospital vendor risk

Medtech breaches keep hitting corporate IT, not devices — and that's a blind spot in hospital vendor risk - Digital Health Insights

https://secure.gravatar.com/avatar/f34ba031cc2ebe143e7fb4aa7b7b7ed3b31f4de1d0d51d659c2c45e2a2a020d5?s=96&d=mm&r=g

Carrie Pallardy

As health care pushes forward with AI, lawsuits like this highlight the tension between innovation and oversight.

Whistleblower lawsuit claims Mayo Clinic's AI puts patient data, safety at risk

Whistleblower lawsuit claims Mayo Clinic&#8217;s AI puts patient data, safety at risk

Whistleblower lawsuit claims Mayo Clinic's AI puts patient data, safety at risk - Digital Health Insights

Google is integrating video analysis into its AI medical system to add audio and visual cues to its diagnostic processes.

What happens when AI diagnostic tools start hearing and seeing patients?

What happens when AI diagnostic tools start hearing and seeing patients? - Digital Health Insights

dhc_network

In 2023, headlines were abuzz with the startling news that an early version of ChatGPT had <a href="https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000198">achieved a passing score</a> on the US Medical Licensing Exam, heralding the imminent obsolescence of human clinicians.

Three years later, human care providers are in no danger of losing their jobs. But more and more of them are being joined by AI colleagues that can augment and optimize the important work that they do – sometimes to an eerily advanced degree.

This growing category of AI clinical assistance tools isn’t just changing the provider workflow. It’s also challenging regulators to rethink traditional methods of assessing software tools for safety, bias, and effectiveness.

Now, the FDA <a href="https://www.fda.gov/news-events/press-announcements/fda-seeks-public-feedback-inform-regulatory-approach-generative-ai-enabled-medical-devices">is considering</a> whether established risk-based frameworks for software evaluation remain sufficient in the era of generative AI solutions – or whether these tools need a more nuanced approach.

In a newly released <a href="https://www.fda.gov/media/194242/download">discussion paper</a>, the FDA highlights the central problem facing regulators: When GenAI can produce a basically infinite number of outcomes based on an uncountable number of variables and calculations, how is it possible to prove that enough of the outcomes will be safe and effective enough of the time?

The answer may lie in treating the software more like a person: establishing an acceptable baseline of competency and continuing to monitor outputs over time to make sure they stay aligned with its defined clinical role.
<h2>The need for a new approach to software evaluation</h2>
In the paper, the FDA explicitly states that “evaluation approaches developed for software with bounded inputs and fixed outputs may not be appropriate for GenAI-enabled devices,” because unlike non-GenAI tools that can generally be held up against a representative set of inputs that produce an expected set of outputs, GenAI simply doesn’t function the same way.

Large language models (LLMs) enable these tools to be highly variable and adaptive, with the same prompt potentially producing different results based on a number of explicable and non-explicable factors.

Therefore, the agency says, “it may be unreasonable to evaluate every conceivable input that the device might encounter during deployment,” the same way it’s not possible to test a medical student on every single type of clinical situation she may encounter throughout her entire career before handing her a diploma.

Instead, “it will be important for manufacturers to consider how to effectively evaluate and monitor the GenAI-enabled device, and the underlying foundation model as applicable, for its specific intended use in a way that can ensure that device accuracy, relevance, and reliability are maintained, once deployed,” the paper says.

Doing so will require a different type of AI regulation: one that could possibly incorporate exam-style testing with supervised piloting, periodic reevaluation, and a defined scope of practice.
<h2>What could a clinician-style evaluation framework look like?</h2>
The paper outlines a possible two-part premarket evaluation system for GenAI tools, starting with the equivalent of a “board exam” that provides a benchmark for safety and accuracy.

The exam would cover five dimensions of competence, including clinical knowledge, analytic capabilities, safety behaviors, communication, and generalizability. For example, the tool would need to sufficiently prove that it can recognize safety issues and appropriately escalate those issues to human users, perform as expected across different populations, and reach accurate conclusions using the most applicable data points available to it.

Upon passing initial tests, potentially conducted by independent adjudicators, the GenAI tools would need to be exposed to some degree of real-world testing. The specific design of the testing may be based on risk, aligning well with the FDA’s existing risk-based approach to medical device evaluation, and could range from limited retrospective analysis to a full shadow deployment in a real-world clinical environment.

Ongoing monitoring may be appropriate after initial market authorization, too, the FDA suggests. GenAI models can experience “drift” due to changing patient populations, data environments, or underlying model components, which may require long-term competency surveillance and periodic re-benchmarking.
<h2>What does “good enough” mean for GenAI?</h2>
The paper presents an interest conundrum around setting guardrails for accuracy. Human clinicians don’t face a strict “three strikes and you’re out” limit when it comes to differential diagnosis, but should GenAI tools have similar leeway to iterate and adapt, or must it present the right answer the first time, every time?

The middle ground may be the most effective way forward.

“For many GenAI-enabled devices, performance might be compared to that of a panel of qualified clinicians whose consensus reflects the applicable standard of care, or to that of a median clinician in practice,” the paper explains.

“With such an approach, the relevant question might be whether the device performs as well as, or better than, a reference clinician on the same task, while recognizing that additional questions may arise regarding whether generalist or specialist physicians should be used for comparison.”

The right comparators for a given use case might depend on how the tool is actually used, including “the combined performance of the clinician and device working together as a human-AI team versus the device working in a fully autonomous workflow, depending on the intended use,” the FDA continues.

However, it’s unclear how this “average clinician” approach would dovetail with emerging questions about legal accountability and liability for harm, or how it might satisfy patient uncertainty around the safety and accuracy of AI tools being used in their care.
<h2>A request for feedback on important regulatory questions</h2>
The paper also serves as a request for industry feedback on these crucial questions about the future of GenAI regulation. Specifically, the FDA is interested in hearing from stakeholders on key issues including:

Is competency-based assessment actually the right framework for evaluating GenAI?

Are there opportunities for the FDA accept greater pre-market uncertainty in exchange for more robust post-market monitoring?

What role should healthcare institutions, clinicians, developers/manufacturers, and independent third-party evaluators play in the evaluation processes? What about AI agents?

What happens if the scope of future modifications for GenAI tools can’t be fully pre-specified during testing?

What responsibility do health systems have after deployment of a GenAI tool? What about vendors who don’t own the foundation models upon which their tools are built? How can the FDA establish accountability without restricting innovation?

Interested parties can <a href="https://www.regulations.gov/docket/FDA-2026-N-7874">submit their feedback</a> on these and other questions through October 19, 2026.

FDA floats "board exam" model for regulating GenAI medical devices, asks for industry comment

Delivering actionable insights, analysis, and tactical information on the most pressing issues facing healthcare professionals.

Home - Digital Health Insights

A new discussion paper proposes testing GenAI devices like med students, comparing them to average clinicians, and monitoring for drift after launch.

FDA floats “board exam” model for regulating GenAI medical devices, asks for industry comment

The Rise of the All-Electric Hospital

How Blanchard Valley Transformed Healthcare IT Operations with a Unified Platform Approach

Explore Analytics

Resources

How a Health System Migrated 32.5M Documents from their Legacy DMS to Epic Gallery in 60 Days

3 Reasons Why FCC Behavioral Health Upgraded its Mobile Device Management

The Latest

Subscribe

Explore

REACH OUR AUDIENCE

Featured Topics