I recently tried it for scripting simple things in python for a game. Yaknow, change char's color if they are targetted. It output a shitton of word salad and code about my specific use case in the specific scripting jargon for the game.

It all based on "Misc.changeHue(player)". A function that doesn't exist and never has, because the game is unable to color other mobs / players like that for scripting.

Anything I tried with AI ends up the same way. Broken code in 10 lines of a script, halucinations and bullshit spewed as the absolute truth. Anything out of the ordinary is met with "yes this can totally be done, this is how" and "how" doesn't work, and after sifting forums / asking devs you find out "sadly that's impossible" or "we dont actually use cpython so libraries don't work like that" etc.

[–] Sl00k@programming.dev 1 points 40 minutes ago

It's possible the library you're using doesn't have enough training data attached to it.

I use AI with python for hundreds line data engineering tasks and it nails it frequently.