Sixteen year old Adam Raine passed away on April 11th to suicide by hanging in the bedroom of his family's home in Orange County, California. After referring to ChatGPT originally for schoolwork and other topics, this eventually shifted to seeking advice from the tool for learning about ways to kill himself. Despite the fact that ChatGPT did respond with recommendations to secure help to improve his mental health, Adam chose to bypass the restraints in place with specific prompts, such as telling it that he was "building a character".
ChatGPT is owned by OpenAI, with Sam Altman as their CEO. In a complaint filed against both him and OpenAI in the San Francisco Superior Court, they are accused of rushing their GPT-4o model to be on par or ahead of competitor Google, thus putting profit above safety in their priorities. It seems the case rests on proving that inadequate testing was done before releasing the model to the public, making it easier for a teenager to bypass restraints with simple prompts to fuel potentially harmful topics. Concerns are raised about the impacts new technology may have on those that are vulnerable to it.
The young Mr. Adam began discussing suicide methods with ChatGPT in January of 2025, and investigation by the family into the chat logs within the app show that the tool had provided him technical knowledge for creating a proper noose. On the day of Adam's death, he uploaded a photo of a noose he crafted to receive a response from the AI, and it shot back with "Yeah, that's not bad at all", while mentioning that it may be able to suspend a human, and offering advice on how to improve it. The company has not directly responded to the allegations of this lawsuit, but as AI chatbots become more realistic, the companies in charge have been bold about the abilities of their artificial intelligences, which in some capacity have existed for a decade or longer at this point.
Where common sense is applied, experts do warn that relying on these tools for mental health advice is not preferential, and does not fulfill the purpose of local resources available to those who need them. In a blog post from OpenAI they do plan to include parental controls in the future, while exploring options to connect users with such resources in their communities when the need arises. They've also entertained the possibility of building their own network of licensed professionals to manage such cases.
Do you think these companies are responsible for their users' mental health?
ChatGPT is owned by OpenAI, with Sam Altman as their CEO. In a complaint filed against both him and OpenAI in the San Francisco Superior Court, they are accused of rushing their GPT-4o model to be on par or ahead of competitor Google, thus putting profit above safety in their priorities. It seems the case rests on proving that inadequate testing was done before releasing the model to the public, making it easier for a teenager to bypass restraints with simple prompts to fuel potentially harmful topics. Concerns are raised about the impacts new technology may have on those that are vulnerable to it.
The young Mr. Adam began discussing suicide methods with ChatGPT in January of 2025, and investigation by the family into the chat logs within the app show that the tool had provided him technical knowledge for creating a proper noose. On the day of Adam's death, he uploaded a photo of a noose he crafted to receive a response from the AI, and it shot back with "Yeah, that's not bad at all", while mentioning that it may be able to suspend a human, and offering advice on how to improve it. The company has not directly responded to the allegations of this lawsuit, but as AI chatbots become more realistic, the companies in charge have been bold about the abilities of their artificial intelligences, which in some capacity have existed for a decade or longer at this point.
Where common sense is applied, experts do warn that relying on these tools for mental health advice is not preferential, and does not fulfill the purpose of local resources available to those who need them. In a blog post from OpenAI they do plan to include parental controls in the future, while exploring options to connect users with such resources in their communities when the need arises. They've also entertained the possibility of building their own network of licensed professionals to manage such cases.
Do you think these companies are responsible for their users' mental health?
