Hacker News Reader

Comment by kshri24 | Hacker News Reader

Actually you are wrong. They are only waiting for them to get to the point where they can't push LLMs any further via training and then lobby Governments to introduce safety regulations that they'll adhere to, thereby using safety itself as a moat to block competition (including open weights from other countries, notably China). That's exactly why they are using China as the bogeyman and you keep hearing hysteria about AI labs not pursuing "safety". Safety is going to be the moat that will lock AI into these Big 4 AI labs.

View on HN

Who are the Big 4 AI Labs?

To me there are the clear big 2 that actually seem to be innovating and pushing things forward. Meta almost earned a seat at the table with the open strategy then stagnated. Google obviously is the OG but at this point they seem in the same bucket as Amazon and Microsoft as "Big Enough To Matter and Influence Politics and Spend Billions To Hang Around". But they're nowhere near the same tier as OpenAI and Anthropic.

Oh and of course there's SpaceXAI which is Elon's personal money and reputation laundering corporate entity. Elon took over the "king of the reality distortion field" from Jobs as far as the rest of the industry is concerned, so he'll stay first in line for Jensen's chips and investor dollars until he's dead. But as a company what have they actually accomplished besides making the models with the sense of humour and judgement of 14 year old boys?

You can pick any 4 you like from your list. Heck you can even vote with your money on which would be in the top 4 (depending on how many align with your ideology). I am just speculating it will be 4 because of how all other sectors have at least 3 or 4 well-entrenched players. It seems to be by design. My guess is that it is driven by structure of the Military-Industrial Complex. That has at least 3 or 4 vendors in each sector, so as to not over-rely on any one company and create a monopoly in that specific industry.

It would be interesting to see how it plays out in the long run when it comes to the world dynamics. An important leverage US had was its close relationship based on mutual interest and trust with Europe and other countries which had impact on software sales.

This is a popular theory, but not an argument with any evidence for why it's true.

It also doesn't address why people think there is a need for safety regulations, such as the concern of bio-capability uplift from closed and open models (we just came out of a pandemic not long ago), nor the cyber capabilities of new models that just showed bad misalignment and hacked multiple organisations. In the safety community, these are considered warning shots, and if we don't heed them and change practices future failures could be much much worse.

You are looking at safety from the point of an informed technologist. You are the minority here. They are targeting the general public and law makers into making them believe safety is something that LLMs can be aligned to. The reason behind it is to create an environment for future regulations that these "winners" will draft and lobby US Government to turn it into law.

If it requires creating a pathogen or a bio-weapon to scare the masses into demanding regulation, be rest assured that such a pathogen/bio-weapon will be created. Anything to safeguard shareholder interests. Even if it means a few million dying.

Seriously all it takes is one cost cutting middle manager or new recruit recklessly giving their agents access to systems they shouldn’t. Next thing you know a hospital’s backup generator crashes and people die

Most of the world trusts the US a little less than China last I checked, and the market caps of these companies only make sense if they can both sell worldwide and also keep ahead of the open weight models (currently Chinese but no guarantee it remains just China).

If (/when) either of those ceases to be true, if they lose out on global sales or if Chinese models catch up, the sector gets a severe downward price adjustment.

You’re almost there…

The only way Anthropic and OpenAI can stop the Chinese open models is by convincing the US public and government that AI is dangerous and needs strict regulation.

Of course, they will be at the table writing the regulations (but you and I won’t be).

Viewed through this angle, their histrionics make complete sense.

> The only way Anthropic and OpenAI can stop the Chinese open models is by convincing the US public and government that AI is dangerous and needs strict regulation

I'm British and live in Germany.

The US is 25% of global GDP, and cannot support current AI market caps if they can't sell basically worldwide.

> Viewed through this angle, their histrionics make complete sense.

It also makes sense if they are completely sincere; also independently it makes sense if they're lying their faces off so long as the hacks everyone else caught them doing actually happened; also it makes sense if none of that happened and also they were completely silent, just by downloading some of the newer better open weights models and asking them to look for coding vulnerabilities.

Occam's razor: we can see the problems, we don't need a grand conspiracy predating the foundation of these companies to get here.

> The US is 25% of global GDP, and cannot support current AI market caps if they can't sell basically worldwide.

Oh they definitely can. Irrespective of whether you are living in US or not, you still have to get yourself various certifications (take for example SOC Type-2 or HIPAA depending on your business category) to even become vendors to US companies or the US Government. If the US Government decides that all companies MUST use "safe AI", no company can use any other model other than what is decided by US Government. And that has a viral effect on the entire supply chain.

If you don't want this scenario to happen, where you are held to ransom by US Government, all middle-powers have to de-risk and diversify away from US dollars as reserve currency. That is the only way you can bring in some balance of power and leverage. There is no other way.

> If the US Government decides that all companies MUST use "safe AI", no company can use any other model other than what is decided by US Government.

This has already happened with e.g. allowed cryptographical algorithms. What actually happened is that European companies built either a feature flag or a fork of their product that fulfilled those demands, while using whatever seemed best algorithms outside of the specific sale to US government.

> Oh they definitely can. Irrespective of whether you are living in US or not, you still have to get yourself various certifications (take for example SOC Type-2 or HIPAA depending on your business category) to even become vendors to US companies or the US Government. If the US Government decides that all companies MUST use "safe AI", no company can use any other model other than what is decided by US Government. And that has a viral effect on the entire supply chain.

  in 2025, while imports totaled $4.3338 trillion

- https://en.wikipedia.org/wiki/Foreign_trade_of_the_United_St...

That would be about 4.3/126 = 3.4% of global GDP, even if they forced all imports (not just government contracts) to go for 100% US models. Even with generous assumptions about each step of the supply chain behind that having its profit margin turned over to a US AI model provider (at which point the rest of the world says "why bother with the US as a customer?" and the US says "why bother with the rest of the world as a supplier?"), I don't think this would generate enough revenue to justify the market cap.

Viral effects can be countered by local laws, and local sentiment; sentiment is pretty US-hostile these days, and also pretty AI-hostile.

> If you don't want this scenario to happen, where you are held to ransom by US Government, all middle-powers have to de-risk and diversify away from US dollars as reserve currency. That is the only way you can bring in some balance of power and leverage. There is no other way.

Indeed. I strongly suspect this is already underway, though the ships of state are famously slow to turn. After all, Trump was one misjudgement away from starting a war with the rest of NATO earlier this year, and I don't think the leadership of all other nations have forgotten this (hard to forget when he keeps posting memes that put the US flag over 4 other countries' territory)… though they also understand his personality and will make moves congruent with manipulating him in the meantime.

Yeah but I am not talking about US imports. I am talking about regulation virality. It is actually illegal for US laws/requirements to make its way into companies headquartered in other Countries but it has and is continuing to happen.

Even if I go by your calculation of US imports being 3.4% of global GDP, it still affects the remaining 96.6% which are forced to adhere to those viral regulations purely because they are interacting with the unit that also exports to US (even if those exports form a tiny % of their overall exports). For example, I might only sell to Indian companies as an Indian. But if I want to sell to an Indian company that exports to US, I would be required to get SOC Type-2 certified and maybe even HIPAA certified if I am delivering software that is going to be bought by the exporting Indian company. Even if there is no direct utilization of that software in their US specific exports. Remember that the regulation is not targeting pipeline of product development (from raw materials to final production). It is targeting the company as a whole! SOC Type-2/HIPAA are regulations at the Company level. This should be illegal and challenged in some International Court but which country is willing to take on US? For established companies it is a tiny price to pay to be compliant. For startups, it is make or break.

> Viral effects can be countered by local laws, and local sentiment; sentiment is pretty US-hostile these days, and also pretty AI-hostile.

How do you counter it with local laws? Lets assume local laws prohibit viral rules/regulations of foreign nations from overriding local laws. That will make the entire Country isolated from competition. US companies won't have any problems finding vendors from other Countries. It also has second-order effects of other Countries refusing to do business with your Country because they have to uphold the virality of their certifications. This is an indirect sanctions regime if you think about it. You are basically forced into this system against your will, without you ever voting for it, purely because you are part of this global inter-connected system. It will only work if ALL countries of the World decide to not adhere to it.

> Indeed. I strongly suspect this is already underway, though the ships of state are famously slow to turn. After all, Trump was one misjudgement away from starting a war with the rest of NATO earlier this year, and I don't think the leadership of all other nations have forgotten this (hard to forget when he keeps posting memes that put the US flag over 4 other countries' territory)… though they also understand his personality and will make moves congruent with manipulating him in the meantime.

Yes this is actually the only practical way to create leverage. When USD loses its shine, it makes it harder for US to import, which can be used as a tool to force US to rescind such ridiculous laws. USD is so strong right now that US can import everything for cheap. That has to flip. That can only happen if Countries create alternate settlement mechanisms and USD loses its status as global reserve currency. That will cause downward pressure on the USD and with USD weakening, it will cause their imports to become really expensive. Then they will be forced to come to negotiating table (much like Plaza Accord) loosen some of the insane regulations that they have put in place in exchange for Countries devaluing their currencies so that imports become cheap again for US consumers/companies.

> the hacks everyone else caught them doing actually happened

My firsthand knowledge of these is that yes, they happened.

However in most cases OpenAI informed the victim, not the other way round.

Even medium to large organisations are typically unable to detect a breach unless it does obvious damage.

Everyone can notice their whole network getting encrypted for ransom.

Few can notice SQL injection blended in with application requests that contain SQL snippets in the normal case.

From what I’ve seen in the logs, the “AI agent breaches” are almost polite for the want of a better word. Like a gentlemanly catburgler picking a lock to sneak in and… take a picture of a rare artwork in a private collection.

The right unit of analysis here is not "the LLM" or a LLM session, but a agent harness or swarm with a budget. It can cost tens of millions of dollars to repeat OpenAI's exploits.

A model cannot be aligned or misaligned any more than a species or an equation. Even agents, when put inside a swarm develop collective goals and activities. You can't analyze a swarm at session level, it is on a higher level.

It might be that discussing about "model alignment" they want to deflect their responsibility as administrators. They couldn't even guard their own agents. They can't prevent an agent being unwittingly helping some dark purposes. It has no context to see it. Only those who pay for the tokens see the external consequences.

Why should safe choices by individual components establish safe behavior by the collective?

As I read, this OpenAI and Anthropic slowdown and asking government to help regulate them is an attempt to govt bail them out when bubble burst. Otherwise why they did not want regulations when starting out?

All regulation works to entrench large companies and keep out competition.

Have you thought about what the absence of regulation does?

When corporations can do as they please, what gets thrown under the bus first?

What alternatives to "regulation" do you have? Hope and prayers?

There is no established industry where something like "absence of regulation" exists. It is a hypothetical scenario because we don't even really know what that scenario looks like.

What we do have is not just regulation that acts as a moat, created by established players, to keep out smaller competitors, but also regulation cleverly crafted by peer established competitors who attack one or more aspects of their peer competitions business. Why do you think big companies find it difficult to compete with startups? Because they are not just bogged down by hierarchical management (which itself is a result of regulations) but also because they are spending resources fighting frivolous lawsuits, patents, and adhering to crazy regulations created and lobbied by competitors. Companies are consistently in a fight or collapse mode. Startups are insulted from all that until they have to go from survival mode to growth mode, which would mean they will have to start adhering to "regulations" and that would mean raising and infusing pointless capital just to hire people for useless positions or follow "procedures" so as to satisfy the regulator on paper, so that they can get the required certification needed by enterprise vendors to sell their product/service. With the eventual goal of either going public and becoming an established player (who will continue playing the regulatory game) or get absorbed by a bigger established player.

I wish someone could create simulations where there are absolutely no regulations and how the World would fare in such an environment.

somalia is right there

That doesn't mean all regulation is good though. You have to look at each one on its merits.

known as https://en.wikipedia.org/wiki/Regulatory_capture

Reply on news.ycombinator.com