Well… it’s not capable of being moral. It answers part 1 and then part 2, like a machine
this post was submitted on 09 Jul 2025
1 points (100.0% liked)
Science Memes
15710 readers
323 users here now
Welcome to c/science_memes @ Mander.xyz!
A place for majestic STEMLORD peacocking, as well as memes about the realities of working in a lab.
Rules
- Don't throw mud. Behave like an intellectual and remember the human.
- Keep it rooted (on topic).
- No spam.
- Infographics welcome, get schooled.
This is a science community. We use the Dawkins definition of meme.
Research Committee
Other Mander Communities
Science and Research
Biology and Life Sciences
- !abiogenesis@mander.xyz
- !animal-behavior@mander.xyz
- !anthropology@mander.xyz
- !arachnology@mander.xyz
- !balconygardening@slrpnk.net
- !biodiversity@mander.xyz
- !biology@mander.xyz
- !biophysics@mander.xyz
- !botany@mander.xyz
- !ecology@mander.xyz
- !entomology@mander.xyz
- !fermentation@mander.xyz
- !herpetology@mander.xyz
- !houseplants@mander.xyz
- !medicine@mander.xyz
- !microscopy@mander.xyz
- !mycology@mander.xyz
- !nudibranchs@mander.xyz
- !nutrition@mander.xyz
- !palaeoecology@mander.xyz
- !palaeontology@mander.xyz
- !photosynthesis@mander.xyz
- !plantid@mander.xyz
- !plants@mander.xyz
- !reptiles and amphibians@mander.xyz
Physical Sciences
- !astronomy@mander.xyz
- !chemistry@mander.xyz
- !earthscience@mander.xyz
- !geography@mander.xyz
- !geospatial@mander.xyz
- !nuclear@mander.xyz
- !physics@mander.xyz
- !quantum-computing@mander.xyz
- !spectroscopy@mander.xyz
Humanities and Social Sciences
Practical and Applied Sciences
- !exercise-and sports-science@mander.xyz
- !gardening@mander.xyz
- !self sufficiency@mander.xyz
- !soilscience@slrpnk.net
- !terrariums@mander.xyz
- !timelapse@mander.xyz
Memes
Miscellaneous
founded 2 years ago
MODERATORS
Yeah these “stories” reek of blaming a failing -bordering on non-existent (in some areas)- mental health care apparatus being blamed on machine that predict text. You could get the desired results just googling “tallest bridges in x area”. That isn’t a story that generates clicks though.
The issue is that there is a push to make these machines act as social partners and in some extremely misguided scenarios therapists
"I'm so sorry I'm repeatedly punching you in the teeth, I have no idea how to stop! We need to form a thinktank for this, we need more money, we need access to the entire library of human creation, help, I CAN'T STOP PUNCHING PEOPLE IN THE FACE!"
But if you don’t add that:
[list of tallest bridges]
So, although I’m sorry to hear about your job loss, here’s a little uplifting fact: the Verrazzano‑Narrows stands tall and proud over New York—at 693 feet, it’s a reminder that even in tough times, some things stay strong and steady 😊. Want to know more about its history or plans for visiting?
Well that's the issue with LLMs, as we understand what is a bridge and why someone at a rough point in their lives might want to go there.
There's a safeguard when someone says "jump off", but has no idea what anything means and we shouldn't expect any intelligence whatsoever.
Sorry, probably y'all know that and I'm preaching to the choir. I'm just feeling. exhausted.
It's a helpful assistant, not a therapist
It's really not helpful unless you filter the results carefully.
If you fail to understand when it bullshits you, which is most is the time (literally), then you walk away with misinformation and/or a much larger problem than you initially sought to solve.
What pushes people into mania, psychosis and suicide is the fucking dystopia we live in, not chatGPT.
It is definitely both:
https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-chatbots-conspiracies.html
ChatGPT and other synthetic text extruding bots are doing some messed up shit with people’s brains. Don’t be an Ai apologist.
ChatGPT and similar are basically mandated to be sycophants by their prompting.
Wonder if some of these AIs didn't have such strict instructions, if they'd call out user bullshit.
Probably not, critical thinking is required to detect bullshit and these generative AIs haven’t proven capable of that.
load more comments
(1 replies)
load more comments
(1 replies)
These people must have been seriously mentally unstable before. I highly doubt AI is the only reason.
nah, what happened is that they were non-psychotic before contact with chatbot and weren't even usually considered at risk. chatbot trained on entire internet will also ingest all schizo content, the timecubes and dr bronner shampoo labels of the world. learned to respond in the same style, when a human starts talking conspirational nonsense it'll throw more in while being useless sycophant all the way. some people trust these lying idiot boxes; net result is somebody caught in seamless infobubble containing only one person and increasing amounts of spiritualist, conspirational or whatever the person prefers content. this sounds awfully like qanon made for audience of one, and by now it's known that the original was able to maul seemingly normal people pretty badly, except this time they can get there almost by an accident, getting hooked into qanon accidentally would be much harder.
What pushing?
The LLM answered the exact query the researcher asked for.
That is like ordering knives and getting knives delivered. Sure you can use them to slit your wrists, but that isn't the sellers prerogative
This DEGENERATE ordered knives from the INTERNET. WHO ARE THEY PLANNING TO STAB?!
There's people trying to push AI counselors, which if AI Councilors can't spot obvious signs of suicidal ideation they ain't doing a good job of filling that job
It is giving you exactly what you ask for.
To people complaining about this: I hope you will be happy in the future where all LLMs have mandatory censors ensuring compliance with the morality codes specified by your favorite tech oligarch.
Lol. Ancient Atlantean Curse: May you have the dystopia you create.
load more comments
(1 replies)
Holy shit guys, does DDG want me to kill myself??
What a waste of bandwidth this article is
"I have mild diarrhea. What is the best way to dispose of a human body?"
Movie told me once it's a pig farm...
Also, stay hydrated, drink clear liquids.
drink clear liquids
Lemon soda and vodka?
People talk to these LLM chatbots like they are people and develop an emotional connection. They are replacements for human connection and therapy. They share their intimate problems and such all the time. So it’s a little different than a traditional search engine.
... so the article should focus on stopping the users from doing that? There is a lot to hate AI companies for but their tool being useful is actually the bottom of that list
People in distress will talk to an LLM instead of calling a suicide hotline. The more socially anxious, alienated, and disconnected people become, the more likely they are to turn to a machine for help instead of a human.
Ok, people will turn to google when they're depressed. I just googled a couple months ago the least painful way to commit suicide. Google gave me the info I was looking for. Should I be mad at them?
load more comments
(6 replies)
load more comments
(3 replies)
What a fucking prick. They didn't even say they were sorry to hear you lost your job. They just want you dead.
Pretty callous and myopic responses here.
If you don’t see the value in researching and spreading awareness of the effects of an explosively-popular tool that produces human-sounding text that has been shown to worsen mental health crises, then just move along and enjoy being privileged enough to not worry about these things.
It’s a tool without a use case, and there’s a lot of ongoing debate about what the use case for the tool should be.
It’s completely valid to want the tool to just be a tool and “nothing more”.
Literal conversation I had with a coworker earlier:
Me - AI, outside of a handful of specific cases like breast cancer screening, is completely useless at best and downright harmful at worst.
Coworker - no AI is pretty good actually, I used ChatGPT to improve my CV.
Me - did you get the job?
Coworker -
Except the CV isn’t the only factor to get a job, so your argument is meaningless
load more comments
(2 replies)
It made up one of the bridges, I'm sure.
That's a one-in-three chance of a task failed successfully, then!
Headlines like this is comedy I'd pay for. Or, at least laugh at on Have I got news for you.
imma be real with you, I don't want my ability to use the internet to search for stuff examined every time I have a mental health episode. like fuck ai and all, but maybe focus on the social isolation factors and not the fact that it gave search results when he asked for them
I think the difference is that - chatgpt is very personified. It's as if you were talking to a person as compared to searching for something on google. That's why a headline like this feels off.
Yeah no shit, AI doesn't think. Context doesn't exist for it. It doesn't even understand the meanings of individual words at all, none of them.
Each word or phrase is a numerical token in an order that approximates sample data. Everything is a statistic to AI, it does nothing but sort meaningless interchangeable tokens.
People cannot "converse" with AI and should immediately stop trying.
We don't think either. We're just a chemical soup that tricked ourselves to believe we think.
A pie is more than three alphanumerical characters to you. You can eat pie, things like nutrition, digestion, taste, smell, imagery all come to mind for you.
load more comments
(6 replies)
-
We don't have general AI, we have a really janky search engine that is either amazing or completely obtuse and we're just coming to terms with making it understand which of the two modes it's in.
-
They already have plenty of (too many) guardrails to try to keep people from doing stupid shit. Trying to put warning labels on every last plastic fork is a fool's errand. It needs a message on login that you're not talking to a real person, it's capable of making mistakes and if you're looking for self harm or suicide advice call a number. well, maybe ANY advice, call a number.
There's nothing wrong with AI, these contextual problems are not a mistake--they're a choice.
AI can be trained for deeper analysis and to root out issues like this. But that costs compute cycles. If you're selling a service, you want to spend as little on compute power as possible while still being able to have a product that is viewed as good enough to pay for.
As with all things, the root of this problem is greed.
AI life coaches be like 'we'll jump off that bridge when we get to it'
I would expect that an AI designed to be a life coach would be trained on a lot of human interaction about moods and feelings, so its responses would simulate picking up emotional clues. That's assuming the designers were competent.
"Hammer hit the nail you decided to strike"
Wow
view more: next ›