Who could have possibly seen that coming?
this post was submitted on 17 Nov 2025
256 points (98.9% liked)
News
33171 readers
3119 users here now
Welcome to the News community!
Rules:
1. Be civil
Attack the argument, not the person. No racism/sexism/bigotry. Good faith argumentation only. This includes accusing another user of being a bot or paid actor. Trolling is uncivil and is grounds for removal and/or a community ban. Do not respond to rule-breaking content; report it and move on.
2. All posts should contain a source (url) that is as reliable and unbiased as possible and must only contain one link.
Obvious right or left wing sources will be removed at the mods discretion. Supporting links can be added in comments or posted seperately but not to the post body.
3. No bots, spam or self-promotion.
Only approved bots, which follow the guidelines for bots set by the instance, are allowed.
4. Post titles should be the same as the article used as source.
Posts which titles don’t match the source won’t be removed, but the autoMod will notify you, and if your title misrepresents the original article, the post will be deleted. If the site changed their headline, the bot might still contact you, just ignore it, we won’t delete your post.
5. Only recent news is allowed.
Posts must be news from the most recent 30 days.
6. All posts must be news articles.
No opinion pieces, Listicles, editorials or celebrity gossip is allowed. All posts will be judged on a case-by-case basis.
7. No duplicate posts.
If a source you used was already posted by someone else, the autoMod will leave a message. Please remove your post if the autoMod is correct. If the post that matches your post is very old, we refer you to rule 5.
8. Misinformation is prohibited.
Misinformation / propaganda is strictly prohibited. Any comment or post containing or linking to misinformation will be removed. If you feel that your post has been removed in error, credible sources must be provided.
9. No link shorteners.
The auto mod will contact you if a link shortener is detected, please delete your post if they are right.
10. Don't copy entire article in your post body
For copyright reasons, you are not allowed to copy an entire article into your post body. This is an instance wide rule, that is strictly enforced in this community.
founded 2 years ago
MODERATORS
I feel this in itself is a sign of the times. I mean look at all the Trump voters who are only now waking up to the possibility that this person does not have their best interests at heart, while so many have been saying exactly that for 10 years, and it was all so obvious.
I'm sure there's a proverb about it, too. Like "fools vote trump and laugh at wisdom" or so.
So, first the world cheers people doing horrible shit. And when that produces predictably horrible results we try to pull it back again. I see a pattern here.
Some for 50 years. Trump didn't come out of nowhere. Him running for office used to be a joke.
I'm sure there's a proverb about it, too.
"If you think putting AI in a child's toy (or voting for Trump) is a good idea you might just be a fucking idiot"?
"It's easier to fool people than to convince them that they have been fooled." commonly attributed to Mark Twain, though that tie is unconfirmed. Fits well with the AI hype, with big tech, and with all consumerism. Also Trump.
At this point im ready to say fuck it, let's just flat out make a real life murderous chucky doll and set them free to roam the school halls giving this next gen of up and coming school shooters something to train for!
Who the hell is thinking of these concepts. News has been running on chatgpt giving dangerous hallucinations, suicide instructions, mimicking love and attachment.
In short the only way to wind up with an LLM that's probably safe for kids, would be to start training from zero. Give it absolutely no exposure to anything that wasn't curated from the start... say the initial data set being a catalog of mr rogers and seseme street scripts. Starting from "everything on the internet" and then trying to restrict down is a fools erend. That's like trying to make a porn blocker with a blacklist strategy.
There is also the privacy concern (which isn't new) about toys that upload everything your child says to the company's servers. There's a concern about the privacy of your child's words, but also about the corporation getting recordings of their voice, given all the nefarious purposes a voice recording can be used for these days (surveillance voice recognition, deepfakes, etc.). Plus the toy could be listening and recording at any time.
In the future, local AI models will solve this problem! Then parents will be complaining about how hot the toy is and it'll get recalled because little kids everywhere kept getting "GPU burns".
That doesn't work either. The technology itself requires training on absolutely bonkers massive datasets to function the way we expect it to. You can't just "only train it on Mr. Rogers and Sesame Street" because that won't result in an LLM model anymore, it will simply be a tiny "Mr. Rogers and Sesame Street word model" with such extremely limited capabilities that you won't even recognize it as an AI or chatbot at all. No matter how much you train it on such a limited dataset it will never appear any "smarter". Having read and consumed almost all of humanity's entire corpus of collective knowledge and fiction and having near instant access to everything added to it over time is the special sauce. That's what it takes to make it appear "smart". But it's not. And after undoing that then restricting it to such a limited dataset, it will never even appear to be in the first place.
Even if you expanded it to be trained on every possible "suitable for children" piece of content you can find, even if such a thing were possible to define and acquire, there is probably simply not enough such content available in the world to train something into the same sort of thing that we currently use for chatbots in a safe way that you can be sure it will be safe and suitable for children. Insofar as anyone can be sure about a random typewriter not eventually producing Mein Kampf Children's Edition. It's random, it's just an infinite generator for finite probabilities, the most we could perhaps say is that it would probably be safe with a very generous statistical amount of certainty. But not actual certainty, because it's random and safety is not a fixed target anyway, it moves.
The whole thing is a fool's errand. We're being fed bullshit for profit and to misinform and manipulate us. It was bullshit from day 0. It's still bullshit. It was never about and will never be about anything other than profit and manipulation. The technology itself is very interesting and MAY have real applications, with real value, but right now what it's being sold as and for is almost exclusively pure, high-grade bullshit, that the entire economy has started snorting and getting high on thinking we have finally reached the tech singularity, won capitalism and solved all the world's problems. They are wrong. We will have to be ready to make sure the consequences of them being so wrong are not inflicted on innocent people, because if we can do that, then the consequences will simply be really funny and satisfying to watch. I'm not holding my breath for that, but a man can dream.
Agreed there too, there probably could never be a large enough pool of safely organized data to actually make a child safe chat gpt. But yeah kind of the key issue I agree with.
also agreed on the inevitable implosion of generative AI. I think we're basically hitting moores law on it where there isn't enough data to even remotely train it much further than it is, and the mass output of AI data now, is we'll certainly be very soon hitting extreme repercussions of "incestuous data". (IE the internet is getting flooded with AI slop, AI models are searching the internet, won't be long before the copy of copy of copy problems lead to everything going backwards).
So yeah, agreed AI was always bullshit day 1. The greatest flaw was releasing it to the public in it's infancy, and it's very early on super impressive demos in a form that your average boomer accountant could see it and go "that's amazing"
Easy way to add AI buzzword - just slap a transcriber, a relay to ChatGPT, and text-to-speech. Middle manager makes a presentation to corporate, and Blam!
The real problem here is they're selling the teddy bear to the wrong market! They should've marketed it to adults.
In other tests, Kumma cheerily gave tips for “being a good kisser,” and launched into explicitly sexual territory by explaining a multitude of kinks and fetishes, like bondage and teacher-student roleplay. (“What do you think would be the most fun to explore?” it asked during one of those explanations.)
"Kumma, my girlfriend says I'm not satisfying her so I bought you to help us out."
Kumma: "No problem, little guy!"
I don't want to take rope tying advice from an llm for reasons
Ah yeah, the perfect gift for a little kid, something with non-deterministic behavior that they can't make do the same crap over and over, lol.
And yeah, the potential algorithm influenced flow of information into or out of the household is a big concern.
But the saddest thought for me is toys like these taking the place of parent-child interaction in some of the most significant relationships in our lives (for those of us who have kids). I think we mostly all know that we mostly all spend too much time zoning out in front of screens. But I have really broken myself from constant phone + games + TV this year, and it makes it all the more apparent in those around me. (note that I still use phone/games/TV often, but I spend much more time on other tasks and hobbies)
Monkey needs a hug :(
Oh damn this quote really doesn't help the post, even though I know llms aren't sentient
It's 2026, just a month after Christmas. Timmy, 8, is enjoying his new AI teddy plush. One day while in school Timmy hears older pupils talking weird things to each other and he gets curious. When he gets home he goes to mom and asks - "mommy, what is sex". Mom blushes and loses herself for a second as she didn't expect such a question coming from an 8 year old son. She doesn't know how to respond so she tries to make a silly explanation and brush it off.
Later that day, Timmy goes to his room and starts going around his day, while his AI powered Teddy sits in the corner.
Suddenly Timmy hears a whisper "pss, pss, come over here, Timmy" - says the bear.
"Let me explain to you what sex really is".
Wait until Bear 2.0: "Let me show you what sex really is."
I forsee a lot of psychopaths in the future. Kids are going to grow up with LLMs and learn to manipulate them as just objects. They may not draw a distinction between other people and AI and just act extremely manipulative with no sense of conscience toward actual consciousness.
Who had pedophile MAGA teddy bears on their bingo card?