Hacker News Reader

Comment by idiotsecant | Hacker News Reader

I feel like a massive blocklist is simultaneously exactly the kind of thing an LLM is likely to hallucinate and exactly the kind of thing you don't want to rely on an LLM for.

View on HN

If they had it build it from public sources then I assume it was combined via tool usage rather than direct generation. Beyond the hallucination risk, the latter would just be a waste of tokens & time.

Modern agent harnesses rely heavily on tools, as well as writing code, to avoid hallucinations affecting deterministic values.

Which means it'd be a lot cheaper, to use tools, and public sources, and just automate it the traditional way...

I take it you’ve never used a modern agent.

I'd rather spend the $0.50 in API credits than waste a couple hours of my time. How much is your time worth?

Well, it'd cost me about 10 minutes, and it wouldn't delete half my data. So... How much is your infrastructure worth?

My infrastructure is build from a git ansible repo that I tear down and rebuild regularly, so $0.

Did you take that time into your account of costs as well?

You're already using automation tools. Seems like you just don't... Want to learn... The tool that would save you hours of frustration, tomorrow or the day after.

It won't. And there is no reason to learn the intricacies of every little script that is written because there is not enough time in the day for that. If the script fails you feed claude the logs and tell it to fix it while working on the things that I actually enjoy working on and matter in the long run.

Well, as it wouldn't be a script, congrats on already losing the skills you would need, to assess whether or not what you set out to do is actually accomplished.

And as we're discussing a security tool... Love to see how you judge it as something that doesn't matter in your stack. Incidentally, mind offering me your IP range?

Maybe you value the few cents (the .50c above seems more like a phrase than an actual value, that much actual cost would be the same as nearly a full days worth of a ChatGPT Plus subscription to combine some lists) over the 10 minutes but I'm not sure I follow why you'd give it access or control of half your data to pull data from public sources on the internet and give you a combined blocklist out of it?

Our current models and harnesses have not been doing a fantastic job, of keeping things confined... The harness defaults to approving random commands, meaning if the model goes for the data, it tends to find the data. And a delete all before replacing it is fine, right?

The sources are public lists on the internet and the result is a combined file you click to download, none of this requires the agent even run on your PC if you don't trust your sandboxes - you can hope the agent deletes OpenAI's cloud instead (even though that's never happened with a billion weekly users).

I'd be more worried about how much you think your time is worth. If I was your customer, and I didn't care about your work being mostly AI made and likely with non-evident flaws, I'd pay you, or anyone with the same mindset, as close as those $0.50 as I could get away with. You can take more jobs to make up for it, as long as the customers don't care.

Even if you hid the fact that your work was mostly AI, the results will make that evident.

How would you as my customer know that my personal work doesn't have non-evident flaws?

> work doesn't have non-evident flaws?

I simplified the double negative to "have evident flaws". I'm not sure I understood the question, because I would know if the flaws are evident.

I applaud you skills then, not making any mistakes that are non-obvious at the time is something that the entire industry wishes they could achieve.

Reply on news.ycombinator.com