Comment by simoncion | Hacker News Reader

Comment by simoncion | Hacker News Reader

Parent Full thread

> Mythos may not be great today but it is not far fetched to imagine...

I...

Look. Mythos was hyped up as the absolute best bug hunting tool ever made... no software was safe from its awesome bug-finding and exploit-writing capabilities. So strong was it that access _had_ to be limited to a select few pre-vetted entities, lest these awesome capabilities fall into the hands of Evildoers(!!!). Mythos' claimed capabilities were absolutely an important part of the "The LLM-based tools we're building are so dangerous that we must have new laws made to regulate us, or else all of humanity is likely to die!" story that the major LLM manufacturers have been building for a while and are telling now.

Now? Not even six months after release? "Well, yeah, okay, it's actually not that great. But imagine how great the next one could be!"... which is the story I've been hearing roughly every six months for what feels like five years now.

As an aside: I often wish we lived in a world where it was illegal for companies to use hype or any other types of emotional manipulation when advertising (or otherwise speaking in an official capacity) about tools that are to be used in a professional setting. Is it anything other than a bare statement of verifiable facts? Big fines, and repeat offenders get jail time. I know it's never going to happen, but it sure would be nice.

Every 6 months without exception people claim that the new generation of tools is absolutely incredible, and the old ones were total garbage, and that you only think they're crap if you were using the old tools. This just gets repeatedly memory holed again and again, and we're expected to always uncritically buy into the idea that they're actually good now against all evidence from the real world

Just for the 'record' - I am totally with you on the hype and just in general the normalization of sleazy behavior surrounding it but I'm not sure we as normal people have any say anymore including where our money is going to be invested.

> I'm not sure we as normal people have any say anymore [in regards to] where our money is going to be invested.

Unless that was your money being invested, and it was a substantial fraction of the total pool of money being invested, was there ever a time when "normal people" had a real say in where the money was being invested?

AFAIK, the only thing "normal people" can do is vote with their "feet" and pick a different prepackaged investment product, different investment company, or take their money and do the investment themselves.

Honestly, this comment of yours seems a non-sequitur and doesn't really address anything I said... you don't have the power to bend investment firms to your whim, but that doesn't mean that you need to -knowingly or not- carry water for the major LLM manufactures by perpetuating the "But think of how great the tools will be in the future!" meme. It has been years now, and everyone who has been paying attention can say with confidence that the LLM-based tools of the future are never that great... they're often not useless, but they're not worth the billions of dollars that have been and continue to be poured into their manufacture.

Related to your "We little people don't have any power anymore!" commentary, I note that TFA mentions that the kernel community has found that these LLM-based bug-finding tools have a false positive rate of ~50%. TFA goes on to mention that Coverity spent a huge number of years trying so hard to get people to buy its automated scanning software that had only a 20% false positive rate, and could not get enough people to buy the software.

Coverity went under because everyone hated how stupid and annoying the tooling was... at a 20% false positive rate. Once the hype machine starts slowing down, no one working at the coal face is going to buy a tool with a 50% false-positive rate. I personally very strongly believe that even if the tools had a 10% false positive rate, no one would pay the actual price that OpenAI and/or Anthropic would have to charge to recoup the research and manufacturing costs of a cutting-edge LLM-based bug-finding tool.

Even after acknowledging that LLMs are fuzzy matchers and have been hyped, I am in the camp that rationally believes they will get better at some things including hunting bugs / chasing security vulnerabilities - my point also is that you and me can do whatever we can but on the grand societal scale we are not going to make a difference as far as slowing LLM adoption down - it just makes sense for some things and there's a lot of people with time and money that will make it do the rest - that's how it works until everything is saturated in the market.

So no I have no way to address anything you said - that was the point, I don't believe you can - not with regulation and not with voting with your money (I am sure some people tried to vote with their money to slow down mega stores and keep the mom and pop shop alive - there maybe some of those still there, but largely it's big chains occupying most of the market) - that stuff hasn't worked - heck LLMs work better today that that stuff has ever.

> So no I have no way to address anything you said...

No, you do.

1) Refuse to spread the "Well, okay, the tools aren't good now, but imagine how good they could be in the future!" meme. And -because these tools are SAAS and the resources allocated to them can be adjusted at any time without warning- evaluate the tools soberly and dispassionately three to six months after release, and ignore 100% of what the manufacturer claims the tools do.

2) Remind people that the major LLM manufacturers have pretty much never not lied about the capabilities of the tools that they produce and sell. Remind folks that the rational thing to do is to ignore the claims of both the manufacturers and boosters and -given that the tools are deliberately designed to aggressively flatter [0] the operator- to evaluate one's interactions with these tools with a huge serving of skepticism.

I get that you feel like there's nothing you can do to change things. But -as I've repeatedly said- you can stop carrying water for these companies by repeating their propaganda.

Assuming that the regulatory-capture and retroactive-immunity [1] gambit the major LLM manufacturers are currently attempting fails, when the fucknormously huge bill for the tools and the fact that so many GPUs are sitting in storage become public knowledge, things will sort themselves out pretty quickly. I'm fairly certain that continued progression of the -IDK- ten or twenty+ lawsuits against the major LLM manufacturers for the crimes they've committed over the years will help push that process along.

[0] ...I think kids these days call this "glazing"...

[1] ...don't believe that Congress would retroactively make obviously illegal conduct legal thereby mooting all in-progress lawsuits seeking justice for that obviously illegal conduct? Go read up on the FISA Amendments Act of 2008.

Don't lower yourself like this. The people working on this are "normal people" too. Don't put them on a pedestal. That's part of the problem here: lionizing people who are actively lying to others to protect their bags and reputation. Stop acting like a pawn for these people. Buying into their delusions of grandeur is what's perpetuating this mess. LLMs are a tool, not a monolith.

I am not lowering myself or lionizing anyone - I am being practical about this. If we as a society have a workable stance for/against LLMs that is beneficial to all of us I am very open to hearing about it - It would however be fair to say that I am pessimistic given a lot of our collective history. And on oop of that nobody seems to be talking about any real solutions either this time around. Just being dismissive or being all-in are equally bad approaches.

> ...nobody seems to be talking about any real solutions...

I guess you didn't bother to watch the video that is TFA. Greg K-H mentions that if you're going to use LLMs to do bug-finding, you should use open-weights models that run locally and "harnesses" built by members of the community You really should watch the video to hear his reasons for why.

There's nothing wrong with LLM-the-technology. There's everything wrong with the major LLM manufacturers.

I wasn't talking about fixing the tools or using open weight models - I don't need to watch anything to know that's an option. You are also now contradicting your earlier stance somewhat by saying what I was saying - LLMs can and will get better - you have now introduced an irrelevant open vs close element which doesn't really matter. There's no way only open weight models and harnesses get better - if you understand even a little, proprietary models with access to same open information but with billions to burn on research and hardware will get better faster.

> You are also now contradicting your earlier stance somewhat...

Not even a little bit, no. Go back and read carefully. [0][1][2][3]

[0] <https://news.ycombinator.com/item?id=49943382>

[1] <https://news.ycombinator.com/item?id=49944036>

[2] <https://news.ycombinator.com/item?id=49951289>

[3] <https://news.ycombinator.com/item?id=49951466>

Ugh stop posting links dude - answer what do you mean by your previous claim for fixing everything you were complaining about earlier - "you should use open-weights models that run locally and "harnesses" built by members of the community" - like how is that not contradicting what you are saying - if open weights and local harnesses can do it Anthropic and OpenAI will do it faster and better. That doesn't make sense at all in this conversation.

> ...like how is that not contradicting what you are saying...

A careful and honest reader notes that I've never claimed or suggested that there exists a software project that will never get better with enough time and appropriately-focused effort.

But, the single thing in this thread that statement by GK-H addresses is your claim [0] that

  ...nobody seems to be talking about any real [alternatives to using LLMs provided by the major LLM manufacturers]...

Of the things I've talked about in this thread, that's the least important one. However, I do understand that it is the easiest one for you to talk about while still staying vaguely "on message".

[0] <https://news.ycombinator.com/item?id=49948725>

Reply on news.ycombinator.com