Are Google AI overviews helping or hindering consumer health education?

Admit it, you’ve searched health questions in Google. Just last week I looked up “what does a croup cough sound like?” when my toddler was hacking in the middle of the night from the latest daycare virus. Feeling unwell drives humans to look for treatment options, peace of mind, and education. Dr. Google is often the first stop on our care journey.     

Google’s latest search functionality is meant to summarize the internet and provide succinct bullet points in hopes of getting you answers faster. Launched in May 2024, AI Overview leverages generative AI to create these summaries above search results. Google hasn’t rolled out AI Overviews for every topic, but if you search a health question, you may see one.  

Anatomy of an AI Overview

What you see:  

AI Overviews appear at the top of your screen when you search in Google Chrome, directly under the search bar. There are links throughout the summary indicating the source information. The sources for “Symptoms of a Common Cold” link to the CDC (Centers for Disease Control and Prevention), Mayo Clinic, Penn Medicine, and Johns Hopkins. There is a tiny disclaimer at the bottom, “This is for informational purposes only. For medical advice or diagnosis, consult a professional. Generative AI is experimental.” 

What you don’t see:  

 According to a May 2024 blog post, “Google AI Overviews are powered by a customized language model, the model is integrated with our core web ranking systems and designed to carry out traditional “search” tasks, like identifying relevant, high-quality results from our index.” Basically, Google engineered their own code that is a mix of a large language model working off of top search results to spit out a hopefully scientifically-backed answer.  

You’ll only find these AI Overviews when you are searching in the Google Chrome browser, using Google as your search engine (aka Google.com.) I tried running the same search of “symptoms of a common cold,” in Safari and the AI Overview only populated in Google Chrome.  

The Health of Google’sAI Overviews is Evolving  

Upon launching AI Overviews in May, Google received a lot of flack from the internet with reports of results recommending that smoking during pregnancy was healthy and that you could add glue to your pizza to make the cheese stick. Google quickly released an explanation for why these abnormal results were occurring and made updates to their algorithms to improve results. Google specifically called out health content in their release and stated that they, “launched additional triggering refinements to enhance our quality protections.”  

Google openly states, “AI Overviews can and will make mistakes. Think critically about AI Overview responses. Explore results from multiple sources to fact-check important info.” With a reported 75-90% of people never venturing beyond page 1 of search results, the stakes for these summaries to be accurate and trustworthy are incredibly high. It is unclear how Google vets the quality of the sources linked in Overviews. A New York Times article examining AI Overview for health said that, “Google declined to provide a detailed list of websites that support the information in A.I Overviews.” That lack of transparency is disappointing.  

However, Google has frequently made public efforts to promote credible information and partner with reputable organizations. During the height of the Covid-19 pandemic, Google prioritized helping people find useful and factual information around symptoms, testing, treatments, and vaccinations. In May of 2023, Google announced a multi-year collaboration with the World Health Organization to provide credible health information to the public. There is a history of organizational will to do the right thing when it comes to surfacing consumer health information, but the latest execution with AI Overviews requires additional work to realize its potential.  

Impacts of AI Overviews on Health Literacy  

 We cannot underestimate the power that Dr. Google has on the world’s population. The US Department of Health and Human Services states that only 12% of Americans have proficient health literacy skills, including the ability to find, understand, and use information and services to inform health-related decisions for themselves. I consider myself highly health literate (if you are reading this article you probably are too.) When I go visit Dr. Google, I am able to distinguish between a reputable source and someone’s personal blog with an MLM supplement agenda. I read a variety of articles and can identify patterns in content to separate fact from fiction. I also have available time to conduct this due diligence and health insurance to cover medical visits when I need clinical advice. These are all huge advantages over the 88% of Americans that struggle with understanding healthcare lingo and make well-informed decisions.  

If done well, Google’s AI Overview could improve health literacy and empower people to care for themselves and their families. The current state of the feature needs work, and I believe that the company has an ethical responsibility to partner with healthcare organizations to ensure that the model is utilizing evidence-based content and not just scraping popular articles on the internet. As the famous AI adage goes, “Garbage in, garbage out.” I am hopeful that over the next few months, Google will adhere to ethical AI frameworks as they improve AI Overview for health.  

Katie D. McMillan, MPH is the CEO of Well Made Health, LLC, a business strategy consulting firm for health technology companies. She is also a curious researcher and writer focusing on digital health evidence, healthcare innovation, and women’s health. Katie can be reached at [email protected] or LinkedIn.