Hacker News Reader

Comment by electrondood | Hacker News Reader

At our company, there is a mad dash to build things using LLMs, so that we can turn to the board and say "see? we have LLMs" so that the board can turn to investors and say "see? we have LLMs"

Some of the use cases are certainly of dubious benefit to the actual product.

View on HN

I'm seeing a lot of the same, and I'd say mostly dubious.

The use cases I've seen that actually make a ton of sense to me are anything with rules engines with cumbersome steps or syntax.

It's so much easier to allow input like 'block traffic from outside the US this weekend from the hours of x to y' or 'junk emails that come from a different smtp server than the domain name indicates' and so on.

Then you need an expert to understand if the output is correct anyway.

Well, you should definitely be able to read the output, agreed on that much.

But sometimes it's annoying to create such rules either due to having to click through multiple screens, or not remembering the exact syntax offhand.

Locally, something like iptables comes to mind. I've used it hundreds of times, but never regularly enough to remember how to write more complex rules without consulting the manual or Google each time.

The way I'd do it would be to have the LLM generate a rule in some fairly-readable rules engine language, and show the rule, examples, and counter-examples for the user to confirm.

This would be like writing email filters in gmail or smart playlists in iTunes.

Possibly, you could ask the LLM for the examples and counter-examples that look most likely to be a bug?

I think realistically, you can't use AI for anything where the output absolutely must be precise. AI is great for when close enough is good enough, but even just specifying "close enough" is hard.

If you need to know that the output is perfect then you need to check it, and if you need to check it then maybe it's better to just get an expert to do it in the first place.

>then maybe it's better to just get an expert to do it in the first place.

Yeah but also maybe an expert gets a shipload more done using an AI? Huge productivity increase? Dunno!

If you're an expert, it's much easier to do the thing rather than to carefully watch over the shoulder of a noob doing the thing.

Again, I'm talking about in situations where you need to be absolutely precise. This isn't for emails explaining simple concepts to a client (which is a good use) but rather a technical document explaining a process to another expert end user.

It seems like both of the examples cited could be done very well with simple code. Is it just a case of saving dev time since you only need to review?

Essentially, yes. A lot of SaaS companies provide their own syntax, workflows, or PL, that you as an end user don't use enough to be perhaps know the exact syntax offhand. Even something like AWS and their policy documents, for example. And don't forget something like AWS/Terraform.

Gotcha. Saves a day of documentation familiarization, essentially.

I used to be cynical about doing business this way, but the more I have dealt with it. It's fun for everyone except the customers.

Investors get a nice presentation. Executives get to be part of the trend for marketing. Employees learn about the latest technology

Glad to hear we’re not the only ones having an emergency LangChain hackathon.

So far what we’ve been able to build feels brittle. But, LLMs are fun to play with though

AI isn't going to fix the fundamental problems of companies.

Note no one is proposing replacing management with AI, just the rank and file.

> ...there is a mad dash to build things using LLMs...

As long as the mad dash translates into healthy 8-hour work days - have fun!

Reply on news.ycombinator.com