Did the author thinks ChatGPT is in fact an AGI? It's a chatbot. Why would it be good at chess? It's like saying an Atari 2600 running a dedicated chess program can beat Google Maps at chess.
this post was submitted on 09 Jun 2025
794 points (92.0% liked)
Technology
71371 readers
3484 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
AI including ChatGPT is being marketed as super awesome at everything, which is why that and similar AI is being forced into absolutely everything and being sold as a replacement for people.
Something marketed as AGI should be treated as AGI when proving it isn't AGI.
Not to help the AI companies, but why don't they program them to look up math programs and outsource chess to other programs when they're asked for that stuff? It's obvious they're shit at it, why do they answer anyway? It's because they're programmed by know-it-all programmers, isn't it.
Because they’re fucking terrible at designing tools to solve problems, they are obviously less and less good at pretending this is an omnitool that can do everything with perfect coherency (and if it isn’t working right it’s because you’re not believing or paying hard enough)
Or they keep telling you that you just have to wait it out. It’s going to get better and better!
why don't they program them
AI models aren't programmed traditionally. They're generated by machine learning. Essentially the model is given test prompts and then given a rating on its answer. The model's calculations will be adjusted so that its answer to the test prompt will be closer to the expected answer. You repeat this a few billion times with a few billion prompts and you will have generated a model that scores very high on all test prompts.
Then someone asks it how many R's are in strawberry and it gets the wrong answer. The only way to fix this is to add that as a test prompt and redo the machine learning process which takes an enormous amount of time and computational power each time it's done, only for people to once again quickly find some kind of prompt it doesn't answer well.
There are already AI models that play chess incredibly well. Using machine learning to solve a complexe problem isn't the issue. It's trying to get one model to be good at absolutely everything.
...or a simple counter to count the r in strawberry. Because that's more difficult than one might think and they are starting to do this now.
load more comments
(9 replies)
load more comments
(14 replies)
well so much hype has been generated around chatgpt being close to AGI that now it makes sense to ask questions like "can chatgpt prove the Riemann hypothesis"
load more comments
(1 replies)
I think that’s generally the point is most people thing chat GPT is this sentient thing that knows everything and… no.
load more comments
(2 replies)
Articles like this are good because it exposes the flaws with the ai and that it can't be trusted with complex multi step tasks.
Helps people see that think AI is close to a human that its not and its missing critical functionality
load more comments
(2 replies)
load more comments
(9 replies)
An LLM is a poor computational/predictive paradigm for playing chess.
Actually, a very specific model (chatgpt3.5-turbo-instruct) was pretty good at chess (around 1700 elo if i remember correctly).
load more comments
(4 replies)
load more comments
(3 replies)
Sometimes it seems like most of these AI articles are written by AIs with bad prompts.
Human journalists would hopefully do a little research. A quick search would reveal that researches have been publishing about this for over a year so there's no need to sensationalize it. Perhaps the human journalist could have spent a little time talking about why LLMs are bad at chess and how researchers are approaching the problem.
LLMs on the other hand, are very good at producing clickbait articles with low information content.
Gotham chess has a video of making chatgpt play chess against stockfish. Spoiler: chatgpt does not do well. It plays okay for a few moves but then the moment it gets in trouble it straight up cheats. Telling it to follow the rules of chess doesn't help.
This sort of gets to the heart of LLM-based "AI". That one example to me really shows that there's no actual reasoning happening inside. It's producing answers that statistically look like answers that might be given based on that input.
For some things it even works. But calling this intelligence is dubious at best.
load more comments
(5 replies)
load more comments
(4 replies)
I suppose it's an interesting experiment, but it's not that surprising that a word prediction machine can't play chess.
Because people want to feel superior because they ~~don't know how to use a ChatBot~~ can count the number of "r"s in the word "strawberry", lol
Yeah, just because I can't count the number of r's in the word strawberry doesn't mean I shouldn't be put in charge of the US nuclear arsenal!
load more comments
(3 replies)
I swear every single article critical of current LLMs is like, "The square got BLASTED by the triangle shape when it completely FAILED to go through the triangle shaped hole."
load more comments
(1 replies)
LLM are not built for logic.
And yet everybody is selling to write code.
The last time I checked, coding was requiring logic.
To be fair, a decent chunk of coding is stupid boilerplate/minutia that varies environment to environment, language to language, library to library.
So LLM can do some code completion, filling out a bunch of boilerplate that is blatantly obvious, generating the redundant text mandated by certain patterns, and keeping straight details between languages like "does this language want join as a method on a list with a string argument, or vice versa?"
Problem is this can be sometimes more annoying than it's worth, as miscompletions are annoying.
load more comments
(4 replies)
load more comments
(1 replies)
A strange game. How about a nice game of Global Thermonuclear War?
No thank you. The only winning move is not to play
load more comments
(5 replies)
Can ChatGPT actually play chess now? Last I checked, it couldn't remember more than 5 moves of history so it wouldn't be able to see the true board state and would make illegal moves, take it's own pieces, materialize pieces out of thin air, etc.
ChatGPT must adhere honorably to the rules that its making up on the spot. Thats Dallas
load more comments
(4 replies)
Ah, you used logic. That's the issue. They don't do that.
Hardly surprising. Llms aren't -thinking- they're just shitting out the next token for any given input of tokens.
load more comments
(2 replies)
All these comments asking "why don't they just have chatgpt go and look up the correct answer".
That's not how it works, you buffoons, it trains off of datasets long before it releases. It doesn't think. It doesn't learn after release, it won't remember things you try to teach it.
Really lowering my faith in humanity when even the AI skeptics don't understand that it generates statistical representations of an answer based on answers given in the past.
The Atari chess program can play chess better than the Boeing 747 too. And better than the North Pole. Amazing!
Neither of those things are marketed as being artificially intelligent.
load more comments
(2 replies)
load more comments
(1 replies)
I mean, that 2600 Chess was built from the ground up to play a good game of chess with variable difficulty levels. I bet there's days or games when Fischer couldn't have beaten it. Just because a thing is old and less capable than the modern world does not mean it's bad.
I'm often impressed at how good chatGPT is at generating text, but I'll admit it's hilariously terrible at chess. It loves to manifest pieces out of thin air, or make absurd illegal moves, like jumping its king halfway across the board and claiming checkmate
ChatGPT is playing Anarchy Chess
load more comments
(2 replies)
Can i fistfight ChatGPT next? I bet I could kick its ass, too :p
There was a chess game for the Atari 2600? :O
I wanna see them W I D E pieces.
Prepare to be delighted. Full disclosure, my Atari isn't hooked up and also I don't have the Video Chess cart even if it was, so this was fetched from Google Images.
I bet that's a slightly unfair representation of what it actually looked like. Graphics back then were purposely designed for how they would look on CRT tvs which add a lot of specific distortions to images. So taking a screenshot of a game running in an emulator without using a high quality crt filter added to the image will be a very untrue representation of what the game actually looked like.
(Don't get me wrong, I'm not saying it actually looked great when displayed correctly, but i am saying it would've looked considerably better than this emulator screenshot)
load more comments
(6 replies)
load more comments
(3 replies)
2025 Mazda MX-5 Miata 'got absolutely wrecked' by Inflatable Boat in beginner's boat racing match — Mazda's newest model bamboozled by 1930s technology.
It's not that hard to beat dumb 6 year old who's only purpose is mine your privacy to sell you ads or product place some shit for you in future.
You say you produce good oranges but my machine for testing apples gave your oranges a very low score.
load more comments
(1 replies)
view more: next ›