Hacker News Reader

> The damage is massive, the world overnight changed narrative from “future value creation is going to be in openai/google/anthropic cloud apis and only there” to a much more murky world. The bottom has fallen out and with it billions of revenue these companies could have made and an attached investor narrative.

My god!! Will someone please think of the ~children~ billions in revenue!

If there was no Linux, how much more revenue would windows / Sun server divisions have made?

And how much poorer would the rest of the world be?

If there was no Linux, it’s unlikely we ever would have had Google, Facebook and Amazon as we knoe it. Free OS was core to build their SaaS.

imagine the increase in GDP!!

"They are the investors of large proprietary AI companies" is just... not true? Not sure where you're even getting this from. I'm a modestly successful upper-middle-class ML engineer, and I've been worried about AI safety since before Facebook, DeepMind, OpenAI, or Anthropic even existed. The most prominent funder of AI risk efforts (Dustin Moskovitz) is a co-founder of Facebook, so if anything he'd be motivated to make Facebook more successful, not its competitors.

Are you the one talking to the European commission though?

Exactly. The moment Sam Altman started talking to Congress about the dangers of AI and how the solution should be only allow licensed companies to develop AI models and that OpenAI should be part of a small board that determines to whom to grant licenses, everyone should have seen it for what it is.

The AI safety cult has some true believers. It's still fundamentally a grift.

So like crypto and web 3;)

so like hedge funds and global finance

This all smacks of the 80's craze against rap music and video games causing violent behavior.

Where is the evidence that access to uncensored models results in harm (that wouldn't occur due to a bad actor otherwise)? And where is the evidence that said harm reduction is greater than the harm caused by the measurable loss in intelligence in these models?

>Primarily due to Mark Zuckerbergs decision to give away a competitive LLM to open source in a classic “if I can’t make money from this model, I can still use it to take away money from my competition” move

I loved it.

Though, he didn't gave it completely away. With Llama/llama2 licenses he has just threatened that he will give it away...

Semantics though: He gave tens of thousands of salviating engineers on the internet the first competitive LLM to play with. Or left the door open for people to take it, if you prefer that narrative. The entire progress chain that has given us ollama, lamacpp and hundreds of innovations in a very short time was set off by that.

Can't agree more on that one :)

I'm far more worried about how they will try to regulate the use of AI.

As an example the regulations around PII make debugging production issues intractable as prod is basically off-limits lest a hapless engineer view someone's personal address, etc.

How do they plan to prevent/limit the use of AI? Invasive monitoring of compute usage? Data auditing of some kind?

I can think of at least a dozen ways to completely ruin the internet or even society using SOTA/next-gen LLMs/GenAIs, we'll be in trouble way before the singularity.

A ton of legit researchers/experts are scared shitless.

Just spend 5 minutes on EleutherAI discord(which is mostly volunteers, academics, and hobbyists, not lobbyists), read a tiny bit on alignment and you'll be scared too.

If you have ample resources, you don't need next-gen LLMs or AGI. You can accomplish this now, without any fancy, hyped technology. Literally, none of the things LLM or AGI could propose or manage to do to harm us is worse than what we can do to ourselves. For AGI, you need a significant amount of resources to develop, train, and use it. To inflict harm, the brute force of a simple human mind in uniform is much cheaper and more effective.

The point is, it greatly reduces the amount of resources needed to do some serious damage, as well as the level of sophistication needed.

You don't need AGI to do damage, current LLMs are already dangerous. IMO, an open-source affordable unfiltered GPT-5 would ruin the internet in a few months.

> ruin the internet in a few months.

I'm sure the internet will be fine, and the web has already been essentially destroyed as the drive for extracting revenue from every human interaction has rendered it just an amusing advertisement for the most part.

Most of the content of the web today is already generated by "bots" even if those "bots" happen to be human beings.

Youtube is rife with AI(edit: this is not necessarily AI) voiced videos of copy pasted wikipedia articles. I find i am blocking new ones everyday. LLM's didn't do that.

gpt5 6 11 90 will exist regardless.

The option where they don't exist doesn't exist, and so it is utterly pointless to spend one second fretting about how you don't like that or why one should not like that. A nova could go off 50 light years from here, and that would kill every cell on the planet. That is even worse than child porn. And there is nothing anyone can do about that except work towards the eventual day we aren't limited to this planet, rather than against that day. It's the same with any tech that empowers. It WILL empower the bad as well as the good equally, and it WILL exist. So being scared of it's mere existense, or it's being in the hands of people you don't approve of, is pointless. Both of those things can not be avoided. Might as well be scared of that nova.

There isn't even a choice about who gets to use it. It will be available one way or another to both good and bad actors for any purpose they want.

The only choices available to make, are who gets a few different kinds of advantage, who gets their thumb on the scale, who gets official blessing, who gets to operate in secrecy without oversight or auditing or public approval.

When you try to pretend that something uncontrollable is controlled, all it does is put the general populations guard down and make them blind and open to be manipulated, and gives the bad actors the cover of secrecy. The government can use it on it's own citizens without them objecting, and other bad guys aren't affected at all, but honest people are inhibited from countering any of these bad users.

Which is a shame because honest or at least reasonably so outnumber the really bad. The only long term way to oppose the bad is to empower everyone equally as much as possible, so that the empowered good outnumber the empowered bad.

Provide a specific example of what you have in mind to further the conversation not just more opining on “dangerous” is my suggestion.

Tailored propaganda, scams, spams, and harassment at a scale that was never seen before. Plugging metasploit into an unfiltered GPT-5 with a shell and a few proxies could be devastating. Undetectable and unstoppable bots would be available to anyone. Don't like someone? You could spend a hundred bucks to ruin their life anonymously.

Each of us could unknowingly interact with multiple LLMs everyday which would only have one purpose: manipulate us with a never-seen before success rate at a lower cost than ever.

At some point AI generated content could become more common than human content, while still being indistinguishable.

Good enough automated online propaganda could routinely start (civil)wars or genocides, Facebook already let that happen in the past, manipulating elections would become systematical even in the most democratic countries.

What already happened in those areas in the last few years, is really nothing compared to what could happen without enough regulation or barriers to entry in the next few years.

What's worse is that all of this, would not just be possible, but available to every sociopath on earth, not just the rich ones.

>> Tailored propaganda, scams, spams, and harassment at a scale that was never seen before.

I believe the state of these subjects right now is already alarming without AGI. You can't exacerbate the horror about the level of tailored propaganda and scams, etc., which you can't even foresee yourself. It isn't quantifiable.

>>Each of us could unknowingly interact with multiple LLMs everyday which would only have one purpose: manipulate us with a never-seen before success rate at a lower cost than ever.

You would build resistance pretty quickly.

>> At some point AI generated content could become more common than human content, while still being indistinguishable.

Oh, there were some numbers on that one. The number of images generated with AI is already several magnitudes larger than the number of photos humanity has produced since the invention of photography. No AGI is required either.

>> Good enough automated online propaganda could routinely start (civil)wars or genocides,

It already does, without AGI. The Black Rock guys say it's good, - war is good for business. You can squeeze the markets, make money on foreseeable deficits.

>> What's worse is that all of this, would not just be possible, but available to every sociopath on earth, not just the rich ones.

But guns available to every sociopath on earth too...

All of your arguments concern how those with malicious intent can harm us further. I would argue that Sam Altman as the sole controller of AGI is a rather unsettling prospect. If only one country possessed a nuclear weapon, that country would certainly use it against its adversaries. Oh wait, that's already a part of history...

> >>Each of us could unknowingly interact with multiple LLMs everyday which would only have one purpose: manipulate us with a never-seen before success rate at a lower cost than ever.

> You would build resistance pretty quickly.

That is adorably naive. The current thrust in LLM training is towards improving their outputs to become indistinguishable from humans, for any topic, point of view, writing style, etc.

But that has already happened. There is no way to distinguish between human and machine-written text.

A squad of marines at Nigerian telecom (or any other country telecom) with access to change BGP routing, will make equivalent harm in under 24h and may enforce month of harms with the changes.

If any middle schooler had the same destructive power as a squad of marines embedded clandestinely in a foreign country the world would be in shambles.

Both can be true: Big companies can lobby for protection and there being risk in the technology that broad diffusion creates additional risks.

Cat's out of the bag though - we're still trading mp3s decades after napster, this ghost won't go back into the bottle and realistically, most of the risks people flag are not AI risks, they are societal risks where our existing failure to regulate and create consensus have already gone past the red line (election interference, etc).

The internet is already being ruined with access to chatGPT, the spammers haven’t even figured out how to use LLama for the most part.

So really, wrong tree to bark up to- the problem is that our existing way of doing things can’t survive AI and you can’t regulate that away as you couldn’t make gunpowder disappear to avoid your city walls no longer working

You seem to make an assumption that the models will only land producers, and not consumers. Why? Asymmetrical compute power? The difference will likely be in size (amount of facts compressed) not capability / ability to detect bullshit.

This said, the trouble is machines may close the gaps in skills faster than we can comprehend and able to adjust. This means quality of life for people may decrease faster from loss of use than it increases from gains (which need to be relatively evenly distributed). This suggests that everyone should own the compute/storage and ability to enhance themselves.

I have no doubt that machines will close the gaps in skills faster than humans will comprehend, however even AGI will have an owner. And if it is Sam Altman, then this dystopian future even more horrible then thousands of hackers running their own AGIs.

Same can be said by a lot of technologies (or pandemics, or climate change). Imagination is a tool - using it for what it can go bad does not seem to be the the most efficient way to use it.

IMO the next gen AI are going to be tiny nukes that middle schoolers could play with on their iPhones.

AI regulation is as needed as radioactive material regulation.

Nuclear energy is great, Hiroshima not so much.

How does that look like in practice ? What do those nukes do?

What's the gist; what are they scared of? Misinformation, and unemployment?

I don't agree with your point, but I love that Facebook released llama into the open. I realized it's not necessarily to undercut their competitors, either. Their revenue grows when high quality content is easier to create. If they commoditize the process of creating content, they make more money. Commoditize your compliment.

High quality content is not a concern for Facebook

>High quality content is not a concern for Facebook [Citation needed]

I'd say it's a huge concern due to its strong correlation with increased usage and thus ad revenue.

For the time I worked there the metric was engagement (with occasional Cares about Facebook intermissions).

One look at newsfeed tells you it’s ad revenue now. Quality has nothing to do with it unless you define quality as clickbait.

In fact, citation needed on “high correlation” unless you take a meta press release which are notoriously misleading. Like 3% of the platform being news

Good enough to share, cheap to create.

such bullshit: to regard a loss of a "potential" as a realized actualized loss....

It’s a direct degradation of investor narrative at a time when money is much tighter.

Nobody says it’s realized loss, that’s not how valuation works.

But Google LRP involves, as one of the first steps, the question of how much money will be allocated to investors (currently with stock buybacks) before other investment decisions, so yes, attacking valuation directly attacks the purse available for aggressive business moves and L&D.

> It’s a direct degradation of investor narrative at a time when money is much tighter.

Uh, no? The investor narrative of "giving away free AI shit" has been in-effect since Pytorch dropped a half-decade ago. If you're a Meta investor disappointed by public AI development, you really must not have done your homework.

That’s not the investor narrative. The investor narrative is choking the competition out of the market and then squeeze the shit out of people. As we see right now in this season of enshittification.

That happens to not work anymore because open source sets a price floor at which people will adopt the alternative.

The investor narrative is always about building a monopoly.

Damaging the investor narrative to your most direct competitor is building in a saturated ad market is an effective indirect attack.

> The investor narrative is always about building a monopoly.

Can you point out how Meta has been applying this philosophy to AI? Given their history of open research, model weights releases and competitive alternative platforms, I struggle to envision their ideal monopoly. You claim that openness is a hostility tactic, but I think Llama wouldn't be public if it was intended to "kill" the other LLMs.

What we've gotten from Meta is more than we've gotten out of companies that should be writing society software, like Microsoft and Apple.

You are misreading my argument. I’m saying Facebook is degrading google and openai investor narrative. If Llama cost hypothetical one billion, they inflict a multiple on that on their competitors with this move while gaining massive technological advantages.

The improvements made to llama by open source community people already have propelled it past Bard by many accounts and this is a model that a few months ago was absolutely non competitive and downright bad.

So it’s a win win. I don’t see the problem

Facebook has been open-sourcing AI research longer than OpenAI has even had the concept of an "investor narrative". I struggle to understand how someone could jump to the conclusion of this being a "scorched earth" maneuver with so many other reasonable explanations. Facebook has a laboratory (FAIR) with a long history of research and releases like this.

> If Llama cost hypothetical one billion, they inflict a multiple on that on their competitors with this move while gaining massive technological advantages.

If Llama cost a hypothetical one billion, then they amortized the cost over the value of the end product and the free advertisement alone.

Maybe their competitors got scooped, but GPT-3 and GPT-4 haven't gone anywhere. Not to mention, there were lots of other language models from FAANG before Llama arrived. It's not like those were made and released to spite their competitors; it was research. Google and Microsoft have lots of open Transformer research you can find.

Inflicting "damage" and gaining massive technological advantages is quite literally not their goal nor what they've done for the past half-decade. If it is, they've done a terrible job so far by collaborating with Microsoft to open their model format and provide inferencing acceleration for outdated hardware platforms.

> The improvements made to llama by open source community people already have propelled it past Bard by many accounts and this is a model that a few months ago was absolutely non competitive and downright bad.

This is something the original Llama paper acknowledged before the community "discovered" it:

> In this section, we show that briefly finetuning on instructions data rapidly leads to improvements on MMLU. Although the non-finetuned version of LLaMA-65B is already able to follow basic instructions, we observe that a very small amount of finetuning improves the performance on MMLU, and further improves the ability of the model to follow instructions.

https://arxiv.org/pdf/2302.13971.pdf

> So it’s a win win. I don’t see the problem

Neither does Meta, nor Microsoft, nor Google, who have all been content to work on progressive and open AI research. Who do you perceive as their "competitors"? Each other?

While I agree that the previous commenter's point is silly, I wouldn't say that anyone should be writing society software. There's no should.