A teenager took his own life in the USA. The mother blames a chatbot for the tragedy, and the company behind it responds with changes.
Character.ai has changed its chatbot offering after a tragic incident. (Source: Netzwelt)
Update from December 17th: The company Character.ai has in a recent Press report stated that it has changed its offering of chatbots. Basically, there should be a separation between the offer for teenagers and that for adults. Adult users can continue to use a less restricted model that allows for a wider range of responses. In the future, filters will be used more strictly for teenagers to block inappropriate or sensitive answers. For entries that mention suicide or self-harm, a pop-up should also appear that refers to aid organizations such as the National Suicide Prevention Lifeline. There should also be greater restrictions on romantic or suggestive content in chatbots for teenagers.
What happened before the changes
In Florida, a case is currently being heard in civil court that seems like a movie plot, but is unfortunately real: a 14-year-old teenager took his own life, and his surviving mother is now suing the company that developed the chatbot in which the boy is said to have had a virtual love affair.
As evidence of the responsibility of the chatbot Character.AI and the accused founders of Character Technologies, Megan Garcia, the mother of the late Sewell Setzer III, and her lawyers cite excerpts from the late teenager's diary. These show that the deceased, according to his own statements, had a love affair with the chatbot. A love affair that is said not to have been good for him.
Don't miss anything with this NETWORK WORLDNewsletter
Every Friday: The most informative and entertaining summary from the world of technology!
