Hacker News Reader

Comment by pram | Hacker News Reader

I've had very mixed experience with LLMs (Perplexity) and sysadmin stuff. More often than not I ask questions for specific CLI invocations and it will literally just make stuff up. This is even after (supposedly) Googling the documentation and analyzing it.

Even today I got terrible advice from Claude about calling Close() on a database in Golang. This kind of stuff would screw over a junior Dev who didn't know better:

"You're right to question this, and I apologize for the oversight in my previous response. Let me clarify: If you're using a connection pool (which is typically the case with sql.Open), you generally don't need to (and shouldn't) call Close() on the *sql.DB object after each operation."

I feel like all AI is doing atm is giving me extreme paranoia from being gaslit so much lol

View on HN

My work is trialing Microsoft’s Copilot for 365. So I have a built in chatbot in teams to ask various questions among other features.

Asking about specific Microsoft documentation it will just immediately bail and tell me to look that up on my own because it was built on training data up to 2022 and may not be fully up to date on the latest documentation. It won’t even link to the page last time I tried. It makes up PowerShell commands and suggested completely out of date options.

So Microsoft’s own AI assistant cannot even provide accurate information about the Microsoft products it is integrated with let alone anything else.

It can be useful for a short script but anything beyond that it is slower and less reliable than doing things myself.

I don't disagree. There's still value in having eg Linux sysadmin skills, because the LLM still doesn't always get to the solution, but I kick off the job and come back a bit later and don't need to try the basic stuff myself, which means I spend more times on the harder problems.

For some reason I don't run into hallucinations as much as other people seem to, assumedly because I'm on well trod paths, but being lied to like that is always a fear. I asked it about argocd and it told me there is a command line program for it and I didn't believe it and had to Google it for myself, which didn't save me time or energy, but I got to ask it how to do things instead of hooting hoping the documentation had the right example.

Except AI was correct this time.

"It is rare to Close a DB, as the DB handle is meant to be long-lived and shared between many goroutines."

source: https://pkg.go.dev/database/sql#DB.Close

No. That was the reply after I corrected it, as in it originally told me to call Close after every *sql.DB handle.

Reply on news.ycombinator.com