Hacker News Reader

Comment by sifar | Hacker News Reader

I get amused that people don't realize that genAI is an existential threat to the internet and everything that has been built on it.

1) One can no longer trust things out on the web. 2) One no longer needs things out on the web.

For 1), I hope the defense mechanism kicks in time to bake security into our computing culture and pervades throughout the stack.

View on HN

> 1) One can no longer trust things out on the web.

I assume you mean software, because we haven't been trusting other things on the web already for decades.

As for software, everybody interested knew about inherent insecurity of supply chain of modern software but the solutions proposed were too expensive. We need an order of magnitude more money lost for organizations to start switching from today's security theater to a model with security built in.

In general and for software in particular too :). For general see my response to ellg.

Even though we were aware of the insecurity of the supply chain, 1) In practice we tend to ignore it except for mission critical cases. We still do. 2) Autonomous vulnerability/exploitation at scale was difficult and reserved for high value targets.

What you said will be accelerated by 2) now.

You were trusting things on the internet before LLM's?

The Internet is quite fine at delivering packages over encrypted channels which I can trust. (Except where interdicted by governments, like in China, India, Russia, Turkyie,..)

The Web is a rather different beast, but the question is not "can you trust the Internet", but "can you trust a random website", and now even "can you trust a previously trustworthy website".

You of course should not trust any pictures or videos as critical evidence, they should be corroborated by other means. But this has been true for several years now.

To clarify, I meant it from a lay person's perspective. I do realize that one can argue if the average person will have developed this awareness now. The difference this time, I feel, is that the genAI tools are widely available for normal people to experiment with which will hopefully help develop this visceral feeling.

Careful system administration and web browsing were relatively safe; nowadays, even upgrading the local libraries carries risk that must be assessed.

It has always been that way. Literally the only distro that encourages an update process with the requisite effort you should be putting in is Slackware. You should be reading the source code you build. You should be building from source. You should fully understand your toolchains. Binary only distros have always been the equivalent of wearing a condom to have sex. Usually fine, but technically outsourcing the hard work to someone that lets be real, 90% never get to know well enough to credibly trust to any degree. NPM & proglang level package management just doubled down on the real-estate you had to shift through.

Being a responsible programmer/sys admin has always been read heavy, as long as I've been alive. Write only code is antithetical to the basis of running a trustworthy system.

> It has always been that way.

The fact that supply chains have always existed is not meaningful. The issue is that the occurrence is considerably increasing. It's factually riskier to administer systems.

> You should be reading the source code you build. You should be building from source. You should fully understand your toolchains.

This is not realistic for the vast majority of the companies.

While there genuinely was fake content and astroturfed material on the web prior to LLMs, the cost to produce this stuff has fallen enormously. A major corporation or a state actor might pay a bunch of money for inorganic content but it was hard for some rando in Estonia to spin up a network of fake content to monetize on tiktok or whatever. This leads to way more fake content about a much wider range of topics.

I can't see an existential threat as in the internet as in it no longer existing. It's busier than ever although maybe with more junk.

> I hope the defense mechanism kicks in time to bake security into our computing culture and pervades throughout the stack.

Not in this world. It would create friction in the money printing machines.

I can't tell if this is satire or not

Reply on news.ycombinator.com