Hacker News Reader

Comment by oblio | Hacker News Reader

> I think the people in DOGE have the skills

Do we know any of them? How many are accountants, auditors, etc, people with decades of experience with government affairs?

View on HN

Even trying to determine who the workers were brought down threats of criminal prosecution and investigations.

With LLMs, it's close to having someone with that experience and knowledge right there with you.

LLMs tend to be very naive in their outputs when you start asking for anything below surface level. If you ask it how to audit something, it'll probably give you a solid high level answer - look at a, b, c and try to build a narrative about how they relate and then look for deviance (I'm not an auditor and I didn't use an LLM for this). Once you start trying to look at the mechanics of how to actually do that, that's when it will start "hallucinating" or just generally swirl. It's the side effect of having a ton of training data on what something is but not much data on how to do it in practice.

This may change at some point in the future, but I would hardly say that using an LLM is "close to having someone with that experience and knowledge," or maybe it is "close" but it isn't a substitute for "having" when dealing with serious topics.

This is an incredibly naive approach to topics that might leave thousands unemployed, uninsured or even dead.

LLMs are basically a C+/B- student, I wouldn't trust my life to any of them.

Knowledge of what exactly?

The entire Internet and whatever context you provide to it. E.g. COBOL, standard auditing practices, step-by-step guidance on what to do next.

I've found that when cross checked against my own expertise, LLMs have dubious "knowledge" at best. Trusting the output with anything you already don't know would just be Gell-Mann amnesia.

Reply on news.ycombinator.com