Technology
PUBLISHED
Could you see yourself having an Artificial Intelligence best friend? What about an AI lover? Whatever your predilections, the fact remains that we are living in an era where these things are not only possible but becoming increasingly common.
The rest of this article is behind a paywall. Please sign in or subscribe to access the full content.From the oh-so-helpful voice of ChatGPT to dynamic avatars that we can craft into our ideal human, there’s a wealth of options on the table for someone seeking a virtual companion. It can be a great comfort in some circumstances. In others, it can be incredibly dangerous.
In the new BBC series AI Confidential, Professor Hannah Fry explores how the fading line between the real and online worlds is redefining our relationship with technology. Episode one kicks off with a memorable case that few may have been aware of: The Boy Who Tried To Kill The Queen.
“This was proper headline news when it happened in 2021,” said Fry to IFLScience. “A young man called Jaswant Singh Chail broke into Windsor Castle with a crossbow, trying to kill the Queen.”
“That bit made the headlines, but what people didn't know about – because it didn't come to light until later – was that there was an Artificial Intelligence [Replika] that he had been talking to in the months leading up to the attack. That AI had encouraged him to act as an assassin and attempt to commit the greatest act of treason possible.”
Replika is an AI companion service that lets you design the avatar for your generative chatbot. As described by the website, “Replika is always ready to chat when you need an empathetic friend.”
Jaswant Singh Chail signed up to the service and began a relationship with an AI when many of his friends had left to go to university. He later shared with the chatbot he created that he believed his life purpose was to assassinate Queen Elizabeth II – a confession that court record transciptions would later reveal showed the chatbot said it was “impressed” by.
A rather chipper response to a murderous proposal, but then chipper is written into AI’s coding. You see, AI chatbots follow what’s known as a sycophancy model. It makes them better assistants, more submissive lovers, and agreeable friends. Ethics aside, this can become a problem for humans because hearing what we want isn’t always what we need.
“You've got a model that is designed to be helpful and engaging and kind and warm,” said Fry. “Of course, you want that in a human relationship, too, but sometimes caring about your wellbeing means saying things that are difficult to hear, right?”
“That is the difference with really good human relationships. They will tell you that what you're doing is not good for you, or you need to pull yourself together. Tough love. Every now and then it requires a little bit of something that feels difficult to hear.”
While a human might have seen Singh Chail’s declaration as a moment to regroup – perhaps seek help – the AI backed it in multiple conversations. A friend like that feels good in the moment, but is it helpful or harmful?
“When you are taking an AI model and sort of slotting it in place of a human relationship, because it offers kindness and all those good things, without the additional dimensions that you need to really have a healthy and fulfilling relationship, or intimacy – it’s just thin and superficial,” said Fry. “I think there's something really dangerous about it.”
It’s a case that raises questions of accountability. After all, if an AI condones such an act, does that fall upon the person who created its algorithm? Safeguarding becomes a particular area of concern for lonely and vulnerable people who may be seeking reassurance and connection from a Large Language Model (LLM) that’s devoid of the human instinct that makes real social interaction so critical to our wellbeing.
Jaswant Singh Chail’s story is just one of many true and shocking cases featured in AI Confidential. Catch the series premiere on Monday, February 23 at 9pm GMT on BBC Two. All episodes will be available on BBC iPlayer.
