What are the cybersecurity risks of LLMs in healthcare?

Large language models (LLMs) are incredibly powerful tools for understanding and processing text-based inputs. They can summarize huge amounts of text, create new written content, answer complex queries, and support natural language conversations with intelligent chatbots.  

They are quickly finding a place in healthcare as organizations search for improved ways to manage their data and provide better experiences for both patients and their providers. But they are not without their risks. 

In addition to challenges around accuracy, bias, and reliability, LLMs can open up potential issues on the cybersecurity front, according to new research from an international team based in Switzerland, Germany, and the United States. The peer-reviewed article is set to be published in RSNA’s Radiology: Artificial Intelligence journal. 

“LLMs can be exploited in several ways, such as malicious attacks, privacy breaches, and unauthorized manipulation of patient data,” the team explains. “Moreover, malicious actors could use LLMs to infer sensitive patient information from training data. Furthermore, manipulated, or poisoned data fed into these models could change their results in a way that is beneficial for the malicious actors.” 

Here are some of the top risks associated with LLMs and their use in the healthcare space.  

• Data model poisoning: Bad actors could intentionally add incorrect, misleading, or malicious data to a training set in an attempt to skew the output.  This could propagate inaccurate biases through the system and potentially affect decision-making and patient outcomes. 

• Inference attacks: Inference attacks attempt to reconstruct sensitive information from partial data included in the model’s training set.  Membership inference attacks could allow a bad actor to determine if specific sensitive data was included in the model’s training data, and this information could then be linked back to a specific individual. Attribute attacks, meanwhile, attempt to extrapolate information about an individual by using fragments of information. For example, knowing that a specific person had a diagnostic mammogram could lead a bad actor to infer that the person has breast cancer and lead to some form of exploitation.

• Jailbreaking: Cybercriminals use jailbreaking to bypass a model’s internal security protocols, which are designed to protect against unauthorized access to sensitive data. The bad actor could use a malicious prompt strategy to extract restricted information or produce unethical medical advice in patient-facing documents, the researchers caution. 

• Prompt injection: Cybercriminals can use prompt injection to override model instructions and force the model to provide information it should not deliver. This strategy could be used to generate inappropriate instructions or recommendations that are not beneficial to the patient. 

• Denial of Service (DoS):  By bombarding a model with a large number of requests, bad actors can overwhelm a model and trigger unexpected downtime.  Legitimate users will not have access to the tool during this time, which can lead to delays in care.

LLMs can also be used to directly carry out cyberattacks at every level of the health system’s infrastructure. For instance, LLMs could be used to create malware programs, including ransomware, that can then be deployed via an LLM-generated phishing email that tricks staff members into clicking a malicious link.  

Bad actors could also use LLMs to create misleading content that influences patients or staff members to make inappropriate decisions, manipulate clinical documents to alter treatment plans, or even generate fake scientific evidence that propagates through the industry. 

Defending against LLM-related cyberattacks starts with paying close attention to the integrity, security, and completeness of training data. Developers will need to ensure that they thoroughly de-identify training datasets, validate and sanitize input prompts to prevent model poisoning, and conduct rigorous testing in a secure and isolated environment to prevent inappropriate intrusion. 

Continuous monitoring and a strong incident response plan will also be necessary to safeguard against the cybersecurity risks inherent in LLM use. 

“As LLMs increasingly become integrated into health care systems, ensuring their security remains a critical concern,” the researchers concluded. “To safely integrate LLMs into health care, institutions must focus on robust security measures and continuous vigilance to prevent exploitation. This requires interdisciplinary efforts across model development, health care regulation, and AI governance to mitigate risks. By promoting the safe, transparent, and fair use of LLMs, all stakeholders in health care can benefit from their transformative potential while minimizing risks to patient safety and data privacy.” 

Jennifer Bresnick is a journalist and freelance content creator with a decade of experience in the health IT industry.  Her work has focused on leveraging innovative technology tools to create value, improve health equity, and achieve the promises of the learning health system.  She can be reached at [email protected].