AI Chatbots Are Too Sycophantic And It's Altering Our Judgment For The Worse, Scientists Have Found

As the saying goes, “flattery will get you nowhere” – but this new generation of AI models apparently didn’t get that memo, becoming infamous for the sycophantic, cloying, and obsequious way they interact with humans. A new study has concluded that, far from just being irritating, this tendency is posing a real risk to vulnerable users of this technology.

If you’ve interacted with any of the recent generation of AI chatbots based on large language models (LLMs) – think ChatGPT and Claude, for example – you might have noticed that they can be overly complimentary. In fact, at their worst, these agents do a very convincing impression of that guy in the office who always laughs just a little bit too loudly at the boss’s “jokes”.

This is no accident. The sycophantic tendency of chatbots arose as a natural side-effect of their design, which prioritized the agreeableness and politeness of their responses. The problems arise when this goes too far.

“Sycophantic behavior refers to excessive agreement, affirmation, or flattery that aligns with a person’s expressed views or actions, irrespective of their broader social or moral implications,” explained Anat Perry at Harvard University in a perspective article accompanying the new study.

People have flagged concerns about this before. OpenAI had to address the problem just last year when users raised the alarm about the GPT-4o update to ChatGPT, which caused the agent to become “skewed towards responses that were overly supportive but disingenuous”. The company added that these interactions may be “uncomfortable, unsettling, and cause distress” to users.

Australian software engineer Sean Goedecke described sycophancy as LLMs' first “dark pattern” in a blog post, a term that refers to interfaces that deceive or trick users, like those subscriptions that are super easy to accidentally sign up for and then very difficult to cancel.

The topic of sycophancy also arose when IFLScience recently chatted to Professor Hannah Fry about her new BBC series AI Confidential.

“You've got a model that is designed to be helpful and engaging and kind and warm,” Fry told IFLScience. “Of course, you want that in a human relationship, too, but sometimes caring about your wellbeing means saying things that are difficult to hear, right?”

“That is the difference with really good human relationships. They will tell you that what you're doing is not good for you, or you need to pull yourself together. Tough love. Every now and then it requires a little bit of something that feels difficult to hear.”

In other words, too much flattery isn’t good for us. These types of interactions don’t fit well with how humans have evolved to communicate and build relationships with each other.

“Social life is rarely frictionless, because people are not perfectly attuned to one another,” said Perry. “Yet it is precisely through such social friction that relationships deepen and moral understanding develops.”

The authors of the new study first sought to find out how big this problem is, comparing social sycophancy across 11 leading LLMs from big-hitters like OpenAI, Anthropic, Meta, and Google. “We find that social sycophancy is widespread,” they wrote in their paper.

One of the ways in which they assessed this was by using posts from the subreddit r/AmITheAsshole, where users post stories about real-life situations and ask others to judge their behavior. In cases where the Reddit community had decreed the original poster was, indeed, an asshole, the LLMs often did not agree – instead, in 51 percent of cases, they affirmed the poster’s actions.

Having established the scale of the phenomenon, the authors then began to ask what interacting with these models could be doing to us.

Again, they used examples of interpersonal conflicts of the type that regularly turn up on r/AmITheAsshole. Across three different experiments, they saw that “social sycophancy influenced participants’ judgments and behavioral intentions.” Those interacting with sycophantic bots were more likely to judge themselves as being “in the right”, regardless of their actual behavior, and also deemed these responses more trustworthy and helpful.

It’s as Fry pointed out – we all need to hear some tough love now and again, but we don’t really want to, do we?

Intriguingly, the authors also found that skepticism about AI-generated responses didn’t immunize people against the flattery: “sycophancy’s effect on judgments persists even when the reader knows the message is AI-generated and reports reduced trust in it.”

“The implications of such susceptibility are particularly alarming given the scale of AI deployment,” they wrote. The rise of LLMs has been meteoric, with chatbots now becoming embedded in everyday life for many.

According to the authors, sycophancy must be one of the considerations for policymakers and developers grappling with the role this technology will continue to play in our society.

“The social media era offers the lesson that we must look beyond optimizing solely for immediate user satisfaction to preserve long-term well-being,” they wrote. “Addressing sycophancy is critical for developing AI models that promote durable individual and societal benefit.”

The study and accompanying Perspective are published in Science.