this post was submitted on 27 Aug 2025
462 points (96.2% liked)

Technology

75117 readers
1918 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

The makers of ChatGPT are changing the way it responds to users who show mental and emotional distress after legal action from the family of 16-year-old Adam Raine, who killed himself after months of conversations with the chatbot.

Open AI admitted its systems could “fall short” and said it would install “stronger guardrails around sensitive content and risky behaviors” for users under 18.

The $500bn (£372bn) San Francisco AI company said it would also introduce parental controls to allow parents “options to gain more insight into, and shape, how their teens use ChatGPT”, but has yet to provide details about how these would work.

Adam, from California, killed himself in April after what his family’s lawyer called “months of encouragement from ChatGPT”. The teenager’s family is suing Open AI and its chief executive and co-founder, Sam Altman, alleging that the version of ChatGPT at that time, known as 4o, was “rushed to market … despite clear safety issues”.

(page 2) 49 comments
sorted by: hot top controversial new old
[–] Occhioverde@feddit.it 3 points 2 weeks ago* (last edited 2 weeks ago) (2 children)

I think we all agree on the fact that OpenAI isn't exactly the most ethical corporation on this planet (to use a gentle euphemism), but you can't blame a machine for doing something that it doesn't even understand.

Sure, you can call for the creation of more "guardrails", but they will always fall short: until LLMs are actually able to understand what they're talking about, what you're asking them and the whole context around it, there will always be a way to claim that you are just playing, doing worldbuilding or whatever, just as this kid did.

What I find really unsettling from both this discussion and the one around the whole age verification thing, is that people are calling for techinical solutions to social problems, an approach that always failed miserably; what we should call for is for parents to actually talk to their children and spend some time with them, valuing their emotions and problems (however insignificant they might appear to a grown-up) in order to, you know, at least be able to tell if their kid is contemplating suicide.

[–] LillyPip@lemmy.ca 1 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

but you can't blame a machine for doing something that it doesn't even understand.

But you can blame the creators and sellers of that machine for operating unethically.

If I build and sell a coffee maker that sometimes malfunctions and kills people, I’ll be sued into oblivion, and my coffee maker will be removed from the market. You don’t blame the coffee maker, but you absolutely hold the creator accountable.

[–] Occhioverde@feddit.it 1 points 1 week ago* (last edited 1 week ago) (1 children)

Yes and no. The example you made is of a defective device, not of an "unethical" one - though I understand how you are trying to say that they sold a malfunctioning product without telling anyone.

For LLMs, however, we know damn well that they shouldn't be used as a therapist or as a digital friend to ask for advice; they are no more than a powerful search engine.

An example that is more in line with the situation we're analyzing is a kid that stabs itself with a knife after his parents left him playing with one; are you sure you want to sue the company that made the knife in that scenario?

[–] LillyPip@lemmy.ca 1 points 1 week ago* (last edited 1 week ago)

Not really, though.

The parents know the knife can be used to stab people. It’s a dangerous implement, and people are killed with knives all the time. e: thus most parents are careful with kids and knives.

LLMs aren’t sold as weapons, or even as tools that can be used as weapons. They’re sold as totally benign tools that can’t reasonably be considered dangerous.

That’s the difference. If you’re paying especially close attention, you may potentially understand they can be dangerous, but most people are just buying a coffee maker.

[–] pelespirit@sh.itjust.works -1 points 2 weeks ago (2 children)

What I find really unsettling from both this discussion and the one around the whole age verification thing

These are not the same thing.

[–] Matthew@midwest.social 2 points 2 weeks ago

Isn't that probably why they differentiated them in the sentence you quoted?

[–] Occhioverde@feddit.it -1 points 2 weeks ago

Arguably, they are exactly the same thing, i.e. parents that are asking other people (namely, OpenAI in this case and adult sites operators in the other) to do their work of supervising their children because they are at best unable and at worst unwilling to do so themselves.

[–] chrischryse@lemmy.world 0 points 2 weeks ago (4 children)

OpenAI shouldn’t be responsible. The kid was probing ChatGPT with specifics. It’s like poking someone who repeatedly told you to stop and your family getting mad at the person for kicking your ass bad.

So i don’t feel bad, plus people are using this as their own therapist if you aren’t gonna get actual help and want to rely on a bot then good luck.

load more comments (4 replies)
load more comments
view more: ‹ prev next ›