Its like having a junior developer with a world of confidence just change shit and spend hours breaking things and trying to fix them, while we pay big tech for the privilege of watching the chaos.

I asked chat gpt to give me a simple squid proxy config today that blocks everything except https. It confidently gave me one but of course it didnt work. It let through http and despite many attempts to get a working config that did that, it just failed.

So yeah in the end i have to learn squid syntax anyway, which i guess is fine, but I spent hours trying to get a working config because we pay for chat gpt to do exactly that....

[–] merc@sh.itjust.works 21 points 1 day ago (2 children)

It confidently gave me one

IMO, that's one of the biggest "sins" of the current LLMs, they're trained to generate words that make them sound confident.

[–] KairuByte@lemmy.dbzer0.com 9 points 1 day ago (2 children)

They aren’t explicitly trained to sound confident, that’s just how users tend to talk. You don’t often see “I don’t know but you can give this a shot” on Stack Overflow, for instance. Even the incorrect answers coming from users are presented confidently.

Funnily enough, lack of confidence in response is something I don’t think LLMs are currently capable of, since it would require contextual understanding of both the question, and the answer being given.

[–] merc@sh.itjust.works 5 points 22 hours ago (1 children)

No, I'm sure you're wrong. There's a certain cheerful confidence that you get from every LLM response. It's this upbeat "can do attitude" brimming with confidence mixed with subservience that is definitely not the standard way people communicate on the Internet, let alone Stack Overflow. Sure, sometimes people answering questions are overconfident, but it's often an arrogant kind of confidence, not a subservient kind of confidence you get from LLMs.

I don't think an LLM can sound like it lacks in confidence for the right reasons, but it can definitely pull off lack of confidence if it's prompted correctly. To actually lack confidence it would have to have an understanding of the situation. But, to imitate lack of confidence all it would need to do is draw on all the training data it has where the response to a question is one where someone lacks confidence.

Similarly, it's not like it actually has confidence normally. It's just been trained / meta-prompted to emit an answer in a style that mimics confidence.

[–] locuester@lemmy.zip 2 points 18 hours ago* (last edited 18 hours ago) (1 children)

ChatGPT went through a phase of overly bubbly upbeat responses, they chilled it out tho. Not sure if that’s what you saw.

One thing is for sure with all of them, they never say “I don’t know” because such responses aren’t likely to be found in any training data!

It’s probably part of some system level prompt guidance too, like you say, to be confident.

[–] merc@sh.itjust.works 1 points 17 hours ago

I think "I don't know" might sometimes be found in the training data. But, I'm sure they optimize the meta-prompts so that it never shows up in a response to people. While it might be the "honest" answer a lot of the time, the makers of these LLMs seem to believe that people would prefer confident bullshit that's wrong over "I don't know".

[–] derpgon@programming.dev 3 points 1 day ago

SO answers and questions are usually edited multiple times to sound professional, confident, and be correct.

[–] film@sh.itjust.works 1 points 5 hours ago

Sycophants.

[–] fibojoly@sh.itjust.works 16 points 1 day ago

Man, I can't wait to try out generative AI to generate config files for mission critical stuff! Imagine paying all of us devops wankers when my idiot boss can just ask Chat GPT to sort all this legacy mess we're juggling with on the daily!

[–] Nalivai@lemmy.world 4 points 11 hours ago

I have a friend who swears by llms, he sais it helps him a lot. I once watched him do it, and the experience was exactly the same you described. He wasted couple of hours fighting with bullshit generator just to do everything himself anyway. I asked him wouldn't it be better to not waste the time, but he didn't really saw the problem, he gaslit himself that fighting with the idiot machine helped.