Hacker News Reader

Comment by stavros | Hacker News Reader

Are they? While I love LLMs, I don't find them extremely useful for much more than faster API documentation.

Or wasting time going through the crap that google throws up in its search results these days. I find it faster to just ask GPT when I forget some command argument. LLMs have basically replaced the web search engine for me in most day to day cases now.

How do you handle the lying/hallucination problem? Do you just run the command and hope?

Anyone who still talks about hallucinations today hasn't used a paid service in the last 6 months.

I just had a paid open ai service tell me all about a command line argument that doesn't exist.

It isn't possible to do what I wanted with the proposed command, but the hallucination helped me to Google a method that worked.

What do you mean? Hallucinations are unavoidable, even humans produce them semi-regularly. Our memories are not nearly reliable enough to prevent it.

In my experience the only more or less reliable way to avoid hallucinations is to provide the right amount of quality information in the prompt and make sure the LLM uses that.

The stuff I use it for is usually to recall or find simple information rather than creating textual content. Stuff like looking up some linux command that I've used before, but can't recall specific parameters/arguments or generating code to get me started on something. So I don't see the hallucination issue much. There have been cases where it has pulled some outdated information tied to a specific library I'm using and I asked it to generate some example code on how to use that library. But with GPT 4 , I don't see that often now.

Now Google's Bard/Gemini on the other hand quite frequently makes up stuff. So for now I'm sticking to GPT 4 via Chatgpt plus subscription to augment my daily dev work.

> Stuff like looking up some linux command that I've used before, but can't recall specific parameters/arguments

So, to repeat the question:

Do you just run the command and hope? Or do you double-check using the manpage that it isn't going to do something drastic and unexpected?

I see what you mean. Yes, I do verify it or do it in an environment where a mistake isn't going to cripple an application or cause some crisis. But in most cases once it points out the argument for the command or the command name - that usually jogs my memory to know that its correct. Lately it's been mostly when creating dockerfiles and also stored procedure syntax. I'm not really good with keeping notes.

I've used them to:

- Make edits to a Latex file which would have taken me at least an hour longer to do.

- Reverse-compile a PDF into Latex from a mere copy-paste.

- Translate a travel diary from French into English.

- Ask conceptual questions in difficult areas of mathematics. It's unreliable but it often has a "germ" of an idea. This is backed up by the Fields medalist Terence Tao.

- Helped me tutor someone by giving model solutions to homework and exam problems when I wasn't sure.

- Write a browser extension to do certain content blocking/censoring that none of the programs on my computer could do. I've never written a browser extension before, and this would have taken me a day longer.

- Give me feedback on emails I wrote.

- Helped me deal with a medical emergency.

Those are good ideas, thanks. I don't like its writing, I find it stilted and awkward, but it's good if you want something that's not going to dissatisfy anyone.

I've also used an LLM for some of the elements on that list, though it seems like I should use them more.

Reply on news.ycombinator.com