😂
ChatGPT answers more than half of software engineering questions incorrectly

You may want to stick to Stack Overflow for your software engineering assistance.

ZDNET

@nixCraft of course it does. To be fair, most developers answer more than half of all programming questions incorrectly :-)

I am not afraid of ChatGPT taking away developer jobs, I am more concerned with leadership that doesn't really understand why ChatGPT appears to be semi-competent at solving their software development problems.

@nixCraft I’m relatively new to Mastodon. I was bracing myself for seeing a reply dog pile from crypto AI maxies, but apparently they’re not here \o/
@cmsj @nixCraft the weird thing is I am starting to suspect that there are actual real people here.
@nixCraft that means that you at least get more answers than on stackoverflow
@nixCraft But was ChatGPT a condescending dick about it? Either way, I can see why the StackOverflow community is against it - who would use SO if ChatGPT was just as wrong and not nearly as rude? #RiseOfTheArrogantRoboticAsswadFromRedwoodCity
@nixCraft Was this with GPT 4 or 3.5?
@nixCraft this is great, we just have to do the opposite of what chatGTP says and we would be correct over 50% of the time.
@nixCraft
Isn't Stack Overflow going into AI? Hasn't it also lost half its user base?
@nixCraft So a bit of software programmed to plagiarize StackOverflow and randomly throw out answers is getting it wrong more often than Stack overflow? Hmmmm…..
@nixCraft At least stack overflow commenters remind you "don't use this in production" when they give you the wrong answer.
@nixCraft ... and it isn't asked the other half of the software engineering questions.

@nixCraft and they say this thing is losing it's owners a horrendous amount of money each day as well.....

🎻

@nixCraft yikes, that was the only reason I would even *consider* using their #AI

Makes this thing dead for most work related purposes imo

@nixCraft 🤔
https://stackoverflow.blog/2023/07/27/announcing-overflowai/

looks like stack overflow went down the drain too
#SALAMI

Announcing OverflowAI - Stack Overflow

@nixCraft @dangillmor Stack Overflow is great for finding condescending 5 year old answers about versions of JavaScript long depreciated.
@nixCraft to be fair SO rate of correct answers might be even worse 😆
@nixCraft Initially yes, but it understands the question much better than google or SO.

@nixCraft no surprise, but I would suspect that the "not clearly wrong" part would be mostly "superficially plausible, and that only to the not actually qualified in the field".

Somewhat related, when I asked the thing to make #OpenBSD #PF configs - https://bsdly.blogspot.com/2023/06/i-asked-chatgpt-to-write-pfconf-to-spec.html or trackerless https://nxdomain.no/~peter/chatgpt_writes_pf.conf.html

I asked ChatGPT to write a pf.conf to spec, 2023-06-07 version

ChatGPT prompt and answer Prompted by this exchange I just had to check what its capabilities were. Here is what happened: Note: ...

@nixCraft

Fact, and it has a very persuasive way of showing wrong answers look right.

Every time I use it I doublecheck the answer.

@nixCraft "stick to Stack Overflow"? Where do you think ChatGTP got it's wrong answers from?
@nixCraft tbf 50℅ is way more than me
@nixCraft Nice try Stack Overflow lol
@nixCraft Says, let's see, looking at notes... 'Stack Overflow!!'
@nixCraft
Too bad stackoverflow will also use AI...