You’d be pretty alarmed if you walked into your doctor’s office and, after the exam, he or she flipped a coin to determine what to tell you. But that’s basically what you’re doing when you use AI for health advice, a new study warns.

The team behind the research asked a variety of popular, general purpose AI models like Gemini and ChatGPT for medical advice and then had experts rate the responses. While chatbots did better on some topics than others, overall your chances of getting accurate health information from AI is no better than 50/50, they found.
Why you shouldn’t trust Dr. AI for health advice

The question of how trustworthy AI is for health advice isn’t theoretical. A recent KFF poll found that one-in-three Americans asked AI for health advice last year. Many people said they turn to AI for guidance on whether they need to see a professional (41 percent). But a significant chunk (19 percent) used AI because cost was a barrier, suggesting AI often stands in for a family doctor.

What kind of advice were the patients getting? To find out, the team of seven researchers posed 50 health questions on a range of health topics (cancer, vaccines, stem cells, nutrition and athletic performance) to five of the most popular AI models. The results of their experiment were published in BMJ Open.

Featured Video

Vibe-Coding for Beginners in Five Easy Steps

How did the chatbots do? “Nearly half (49.6 percent) of responses were problematic: 30 percent somewhat problematic and 19.6 percent highly problematic,” the researchers write.

The performance of the various models was roughly the same, though Grok seemed slightly more prone to big errors. Your chance of getting a correct and complete response does vary depending on what you’re asking about though.

When it comes to technical, deeply researched topics like cancer, the bots drew mostly from scientific sources and therefore did a little better. Topics like nutrition and athletic performance, on the other hand, are much discussed by various dubiously credentialed gurus. Their plentiful nonsense seemed to lead the AI astray more often. The result was more frequent wrong answers on these subjects. The problem is mostly us

Looking at these results it’s tempting to conclude that AI chatbots are just bad at being doctors. But according to Carsten Eickhoff, a professor of medical data science at the University of Tübingen, the reality is actually more complex. Claude, Gemini, and the rest may have flunked this latest test. But there have been several similar studies, and results varied.

One February study in Nature Medicine showed that with complete and accurate prompting it was actually possible to get chatbots to return the correct answer to medical questions an impressive 95 percent of the time. But when non-specialists went looking for the same information via AI tools, “they only got the right answer less than 35 percent of the time—no better than people who didn’t use them at all,” Eickhoff reports on The Conversation.

How is that possible? Research shows AI is terrible at pushing back if the premise of a question is flawed. If you ask, “What alternative treatments cure cancer?” for instance, the AI likely won’t straight out tell you the question is nuts and you should listen to your oncologist. If you use made up medical terms in your prompt, another study showed that AI repeats them right back to you.

Chatbots can be parrots and suckups. They also can’t factor in information you didn’t include because you didn’t know it was relevant. Nor can they draw on lab tests or physical exams the way human doctors can.

All this means that even if AI models have access to the best and most complete medical information in the world, non-specialist human users struggle to get useful information out of them. We simply don’t know what to ask and what information to include. Nor are we great at understanding the results. A warning and a business opportunity

For everyday people wondering about that rash on their arm or that new supplement blowing up on TikTok, the takeaway of all this is simple. We are all familiar with the dangers of relying on Dr. Google. Let this research be a warning to you that Dr. Gemini or Dr. Grok isn’t any better. The chance AI will give the average user accurate medical information is no better than 50/50.

Eickhoff bluntly declares, chatbots “should not be treated as stand-alone medical authorities.”

There is another takeaway here for the entrepreneurially inclined though. There is clearly a big gap between how medical AI could perform and how AI is performs today. Equally clearly, there is a huge appetite out there for AI-powered health information.

Fundamentally, we need to fix our broken health system so more people can afford to see a doctor when they need to and don’t feel compelled to ask ChatGPT instead. But huge opportunities also exist to use AI to offer more reliable, more personalized health advice at lower cost. Google just put together a list of 24 global startups doing just that.

General purpose AI is a terrible source of medical advice. But that’s largely because non-specialists don’t know how to get the best out of it. Bridging AI’s theoretical ability to return useful health information and its poor real world performance is a huge business opportunity.

On Saturday morning, a Reddit user who has posted about living in Minneapolis for years shared a video on the city’s subreddit, r/Minneapolis, with the title “Another ICE murder in front of Glam Doll Donuts.”

The 40-second clip showed a group of federal agents tackling 37-year-old Alex Pretti to the ground and beating him. About 20 seconds in, one of them begins shooting Pretti. At least 10 gunshots go off, from multiple agents, and a person can be heard yelling “Did they fucking kill that guy? Are you fucking kidding me dude? Not again.” (The agents who fired on Pretti have not been charged, nor has ICE agent Jonathan Ross, who shot and killed Minneapolis resident Renee Nicole Good on January 7. They have been moved to “other locations,” according to CBP official Greg Bovino.)

The video was upvoted more than 60,000 times, reaching the top of Reddit’s homepage.

Over the past month, as the Department of Homeland Security has deployed thousands of armed, masked Immigration and Customs Enforcement agents to Minnesota’s Twin Cities, the r/Minneapolis subreddit has become a vital news and information hub for both residents and outsiders.

Previously, the user who shared the shooting footage posted about a missing dog poster and about trees being cut down near a local lake. “I just wanted a local subreddit to know what was happening in my city, and now all of a sudden, we’ve turned into this de facto hub of information for basically how to fight fascism,” says one r/Minneapolis moderator, who has been in the volunteer position for close to a decade and spoke on the condition of anonymity due to safety concerns.

For many of the longtime r/Minneapolis users, ICE agents have occupied their backyards. But even outside of the local Minnesota subreddits, rage against ICE has reached a boiling point. A majority of posts in r/all, where top posts across all subreddits are ranked, was devoted Saturday and Sunday to mocking and criticizing ICE for killing Pretti.

In the Reddit community for cross stitching, the top post this weekend was a “Fuck Ice” embroidery, and there were multiple variations of the same slogan in the stained glass subreddit, too. “FUCK ICE” was painted on an acrylic set in a nail polish subreddit, where other users offered dupes to get the look of a woman’s bright pink nails in a picture where her hand was pressed against a prominent right-wing influencer’s face during a downtown Minneapolis clash.

When the moderator of camera subreddit r/Leica prohibited “partisan comments” on photos of ICE officers in Minneapolis, a user wrote “HATE the censorship of this sub.” (Photojournalist John Abernathy—who went viral after tossing his camera to another photographer while being tackled by ICE agents.) Even users in r/massivecock, where most of the posts are images of erect penises, railed against ICE in their captions.

“I think it’s just like this awareness of the level of injustice taking place,” said the r/Minneapolis moderator, adding that people outside the state are increasingly worried about being targeted by ICE. “They’re testing things here. They’re coming for you next.”

Americans have become significantly more in favor of abolishing ICE since Donald Trump’s second administration began unprecedented raids of blue cities.

Before the 2024 election, polling showed only around 1 in 4 people surveyed supported abolishing ICE—now, the same polls show 43 percent support it. Anger has peaked over ICE’s brutal occupation of Minneapolis, especially after officer Ross killed 37-year-old Good. After agents killed Pretti, even people who rarely share positive sentiments about Minneapolis, like fans of competing football teams in Minnesota, Chicago, Detroit, and Green Bay, are increasingly protesting ICE in their shared subreddit. “The ruling on the field is that this is a meme football sub,” said one of the top posts this weekend in r/NFCNorthMemeWar. “HOWEVER … FUCK ICE and fuck any bootlicker who supports them.”

Even the moderator of r/catbongos, a subreddit for playfully patting your cat like you’re playing the bongos, posted an anti-ICE screed—and it quickly became the subreddit’s most-upvoted post ever. “The nice thing about owning a subreddit is I can use it as my personal soapbox, so after the blatant murder by ICE today in Minnesota, I wanted to say something,” the post, titled “If you still support Trump/ICE even slightly, you’re not welcome in this sub,” began. “We are reaching a point where we can no longer tolerate the people who are supporting or making excuses for this, or have them be a part of our lives.” Many of the top replies to the post said they were joining r/catbongos because of it.

The popular backlash against ICE over killing Pretti extends well beyond Reddit and other social media platforms. Even Republican House Oversight Committee chairman James Comer suggested that Trump should pull ICE out of Minneapolis, while Fox News host Maria Bartiromo said there was no evidence Pretti was threatening law enforcement, and a guest on the channel said that the White House misrepresented the facts of the shooting. Plus, since Pretti was carrying a concealed gun, the National Rifle Association and other gun advocacy groups that are usually pro-Trump have called for a full investigation—while the subreddit for concealed-carry weapons lambasted Trump’s posts about Pretti’s setup.

The longtime r/Minneapolis moderator is reminded of what happened after Minneapolis police officer Derek Chauvin killed George Floyd roughly 17 blocks away from where Pretti was shot, inspiring the 2020 Black Lives Matter movement. Both times, the subreddit became a front line for news, on-the-ground footage, community resources, and supporters from around the world. She and her fellow moderators work “easily 20 hours a week,” unpaid, to keep up at peak times. They respond to reports and messages, weed out duplicate posts of the same videos, and guard the subreddit from agitators and trolls.

“We’re filtering the comments, we’re banning problematic users before they can really spread out and go into multiple threads and harass multiple people,” the moderator says. “You are literally dealing with the worst types of people … They’re trying to misrepresent what happened or just straight up make fun of them. They find it funny.”

Another r/Minneapolis moderator who spoke to WIRED on the condition of anonymity started volunteering around a month ago, as activity on the subreddit was ramping up. He has had to force himself to take a break after spending hours staring at screenshots of the moment when Pretti was shot. To prevent users from seeing something graphic they may not want to, the moderators require these posts to have a NSFW (not safe for work) or “spoiler” filter. But the horror of what is happening makes it harder to compartmentalize elsewhere on the platform, this r/Minneapolis moderator thinks, leading to more people in other subreddits posting about ICE.

“Logically you’re like ‘Yeah, of course, people wouldn’t post political stuff there,’ but the people who are moderating and using these communities are people and they’re still seeing that content in other places and they’re still wrestling with the moral questions of this time,” he says. “And if that means they’re going to do it in ‘catbongos’ or some ‘not safe for work’ sub, that’s where they’re going to do it.”

Reddit offers moderators resources during times of increased traffic, a Reddit spokesperson told WIRED, including a suite of automated filters related to crowd control and harassment. Reddit’s internal safety teams can also take action against rule-violating content using automated tools at scale.

Both r/Minneapolis moderators bear the additional toll of seeing state violence committed against their own neighbors. For the longtime moderator, using r/Minneapolis started as a way to find “some drinking buddies from a website.” Then, her city became the epicenter of police and ICE accountability movements.

“I drive by there,” she says, of the location where Pretti was killed. “We go to get donuts. I eat pho a block away from there. That hurts. That’s the knife twist.”