Hacker News Reader

Comment by 01100011 | Hacker News Reader

Humanity has discovered a way to create a form of intelligence using math. This knowledge is not going back in the box.

But that math cant run without massive GPU clusters. We don't have to allow openai or anthropic access to those anymore than we have to allow a company to operate nuclear power or a bank.

So you are contending that individuals cannot run advanced models? What brought you to that conclusion?

Secondly, are you contending that progress in model efficiency and hardware just stops at whatever level you think is sufficient to prevent individuals or organizations from acquiring sufficient resources to run advanced models?

I think it's more the noise, power consumption, local water consumption, and wholesale theft of creative works.

..."government of the people, by the people, for the people, shall not perish from the earth." -Lincoln, Gettysburg Adress

Unfortunately for AI, it still is. People still get to decide things at the city, town, village level.

No. And that’s not necessary for my position. I have 4 Mac studios. I run large models and am building propelcompute.com to let people self manage clusters of their own hardware or combine hardware to run models. It’s not the models that are the problem. It’s that the people building them are shielded from the consequences of what they are building. I would say if you self host a model and it takes down a power grid you are personally liable for the consequences. I believe in broad distribution of AI and advancing its capabilities but not in a manner that socializes the harms and privatizes the gains which is what we have now.

Absolutely. Any AI could have a "legal team" (or conscience) that checks if any output and performed actions are permitted according to local (server location and client location) laws, so not violate human rights, Asimovs laws etc.

It's just that so far nobody cares about explicit checks because they cost resources or slow down the models.

Ok but this response has nothing to do with your other comment. What exactly were you trying to say?

Just because I can run near frontier level open weight models doesn’t mean I can continue to train models of equal or superior performance, doing that requires massively more hardware. And even if I did, that doesn’t mean I can serve those models to millions or billions of users.

My point is that I would rather have 1000 labs training and serving inference than 2 because that would distribute the wealth creation more broadly rather than allowing OpenAI and anthropic to capture all the value, it would drive more innovation as a broader set of experiments are pursued in parallel.

Again you're shifting your arguments and not addressing your initial response.

How am I shifting my argument. They are corollary arguments. Massive data processing is necessary for frontier model training and inference. If OpenAI and anthropic are behaving irresponsibly they we can limit their access the level of compute to train and serve models. That doesn’t imply individuals can’t run frontier level models or should be prohibited from doing so. That’s a coherent position across a spectrum of possible scenarios: I look at the behavior of the people running the model and would limit their ability to train and run models if they act in unsafe ways.

Who is "we"? The GPUs and training algorithms get more efficient all the time. In a few years, creating effective LLMs isn't going to require massive GPU clusters.

We is society through government via regulation. I don’t think GPUs or training will get that efficient that fast absent a distillation target provided by the easily accessible frontier lab APIs.

Further, even if you are right, so what. Is that a reason to just accept bad public policy? That’s like saying, anyone can learn how to make smallpox at home with a basic lab set up so we should just ignore any safety measures.

What a silly comparison. LLMs have nothing to do with smallpox.

Computing always gets cheaper and faster over time. We can argue about the exact rate of improvement but the results are inevitable and uncontrollable.

Short reminder from the guidelines:

'When disagreeing, reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3." '

Also LLM's have something to do with smallpox as a unrestricted LLM will happily guide any wannabe terrorist in how to make them.

But the amount of available compute has been contractual bought by the frontier labs such that you can’t get equivalent compute even if you had the money. That is market lock up and is a policy choice. Free markets require market access. Anticompetitive contracting destroys markets and innovation. We don’t allow that in any other commodity market and we shouldn’t allow it for GPUs. You are not allowed to legally corner the market for silver or soybean futures.

I am reasoning from analogy. Smallpox is dangerous so we have regulations that limit who can do research and how they do it. If LLMs are similarly dangerous we could do the same. As the government did with mythos.

Nah. There is no actual evidence of such danger. Fears about Mythos were completely overblown.

> Is that a reason to just accept bad public policy?

Not necessarily, but it should probably inform that public policy. I think the problem is no one knows what the public policy should be assuming that scenario is true. Even if you, somehow, regulate away massive GPU cluster training making such future training impossible, existing models are already here. Further already training smaller models for things like images, speech, and other specialties is cheaper than the bigger models.

I agree that we need some regulations like everything else, but it’s not clear to me what the right policy should be. I think the European ai act is a fine start, but it’s clearly not enough nor does it necessarily limits the training portion just the application portion. Not to mention that the requirements there can be summarized into something like “you have to be careful, and show evidence you tried to be careful”.

“you have to be careful, and show evidence you tried to be careful”.

That sounds reasonable. If applied to OpenAI and their agents multiple times breaking out of bad secured sandboxes, it should be enough.

But limiting the training?

There really is china and they have a different approach I suppose. But it is possible to talk with them.

> That sounds reasonable. If applied to OpenAI and their agents multiple times breaking out of bad secured sandboxes, it should be enough.

Does it? To me it seems reasonable for OpenAI to argue they did try to be careful evident by the sandbox, they just made a mistake. Almost every 0day is categorized by something like that. We haven’t had a long history of establishing a negligence charge to security bugs. Could you be sued because you didn’t demonstrate “carefulness” and used Linux which is not written in a memory safe language and has had multiple CVEs before? How complicated should the chain of an exploit be to demonstrate “carefulness” to the courts?

> training

OP was the one suggesting that training could be controlled because massive gpu clusters could be regulated the way a nuclear power plant could. If you assume training costs won’t drop, then it’s feasible I guess. However, unlike a nuclear reactor, the final training result isn’t a radio active material, but rather an ordinary file that anyone can load and use for inference.

It seems really clear at this point China has only been able to keep pace by distillation farms.

My entire point is that we are currently on a path to a duopoly which isn’t just expected to cover training and serving models but the entire knowledge economy. That could be mitigated by limiting their train and inference capacity to a specific percentage of total available compute. That would ensure other operators could compete in the market. Instead we are letting OpenAI literally contract to buy all available ram to the point that Apple can’t buy ram and had to cut their hardware configurations.

Apple could have bid higher but chose not to. Most modern desktop software is extremely memory inefficient. Developers have largely ignored this in recent years because RAM was so cheap. But there's tremendous opportunity for improvement with a little optimization work.

We used to run Microsoft Word and other popular applications with 8 MB RAM and it worked fine.

No. Once the contract is signed it will be breach of contract. It’s not always a bidding process.

I am working on reducing ram requirements to run models but the weights are already compressed to the edge of the Shannon entropy boundary and might resist further compression.

> cant run without massive GPU clusters.

How far back into the history of computing do people who keep repeating shit like that know about? God.

Look at the thing in your fucking hand. Now go back just 20 years and see how things were.

For anyone else curious:

> In 2006, the mobile phone market was dominated by stylish flip phones, early music players, and physical keypads just one year before the iPhone changed the industry

Oh my god, I feel so old lol

I miss phones with real keyboards so much.

Just because monopolies die over time doesn’t mean they aren’t destroying value while they exist.

So we should just yolo speed run this because of Moores law? How about you recognize there is a set of rules outside of tech and we can decide how to define them.

Just because models and GPUs will be more advanced in the future doesn’t mean we need to let OpenAI and anthropic establish monopolies on the backs of stolen training data give unfettered access to the internet, the terminal and people’s file system while also allowing them to have limited liability protection behind the corporate veil. That’s a choice.

> Just because models and GPUs will be more advanced in the future doesn’t mean we need to let OpenAI and anthropic establish monopolies

Exactly, again, look at what COMPUTERS THEMSELVES used to be in the 1960s/1970s.

What the "P" in the PC stood for and why it was such a big deal

Which is why I am building propelcompute.com to let people run and manage their own inference clusters and band together to create coops

But who are "we"? The society, the government, the regulator, or the consumer?

I don't see any of such entity would solve that problem. The government and regulator are in OpenAI and Anthropic's pocket, and I don't trust them a single bit on coming up with regulations. The consumers don't care; they just need something smart and cheap. And the society doesn't work either: each person is too busy fighting for their own survival rather than changing the system.

People thought the same in the guilded age: standard oil is too big, Edison and Westinghouse already have the electric market captured and have captured political power. It was a similar transformation in society. If all we do is follow the same playbook America did then we’d be in a better position that the current approach.

There are clear anti-trust mechanisms to prevent market capture and the emergence of asymmetric power. Go back and see how much nashing of teeth Lina Khan triggered in SV when she started to enforce antitrust law and then compare it to what the Pinkerton agency was doing in the transition from the guilded age to the progressive era.

There is a vocal segment of SV that wants the return of the guilded age. Marc Andreessen as said that explicitly. Those of us in SV that value free markets and recognize that the progressive era actually saved markets from their natural tendency to self destruct when winners capture markets and destroy competition that provides the incentive to innovate and drives the price setting function for efficiency.

But if you are an aspiring and ambitious founder in Silicon Valley, which side would you take? The Glided Age mentality, or the Progressive Age mentality? The VC playbook made it very clear that the role of startups, or at least the very successful startup, is explosive growth, expanding quickly, capturing a monopoly and its distribution. Even if you are thinking otherwise, soon you'll face the reality of Silicon Valley's VC growth machine.

In other words, the aspiring founders look at Google, Facebook, Tesla, etc. and think "how can I become one of them", not "how can I be different from them"? If you can find me examples of successful startups that think fundamentally different from those big techs, I'm very happy to be wrong and be corrected here.

> ...massive GPU clusters. We don't have to allow openai or anthropic access to those anymore

if anybody was looking for a good reason for datacenters in space.

https://orbitalradar.com/regulatory/who-regulates-space

You can block their use of domestic airspace.

The knowledge how to build an A-bomb is also not going back in the box.

It only takes me excavating massive amounts of uranium ore, building huge facilites packed with thousands of centrifuges that span multiple square miles, and paying all that infrastructure and workforce.

Your proverbial genie can be out of the bottle all you want, but it doesn't work without getting kicked in the ass by a very large golden boot.

Bomb manufacturing doesn't scale with Moore's law.

And everyone had a fairly good idea what fission and fusion bombs would do once built. (Teller was worried Trinity might set off a nitrogen fusion reaction and kill all life on Earth, but Bethe and others proved him wrong before testing.)

No one knows what the limits of AI are. It's not just untested, it's unmodelled, and unplanned - build it first, worry about consequences later.

Like farming, engines, computers before it.

There is lots of knowledge that requires a license to operate commercially. We could just do this.

So you cede the frontier of human progress to illicit organizations and other nations?

These LLMs are not the frontier. They are a tool. A tool that doesn't need to be able to write a sonnet to be useful.

We don't know that. We don't know if this particular kind of tool can do "useful" things without also developing the ability to write a sonnet. And they are absolutely a frontier. I am not suggesting we should continue building them just because they are, there are many technologies which could have been built had we thrown the amount of resources we have here and there is a case to be made for not doing it at the pace we are or if at all. But we can do that without diminishing what exists.

I just mean the business case is much more likely to come from treating it like a tool than a proto-diety. Frontier of business, not frontier of impressive demo. Also, would argue they aren't frontier of research, in that a research frontier expands in more directions than just the latest hot thing in silicon valley. See also, LeCun, Chollet, etc. "These labs aren't the entirety of the frontier" would have been more accurate. And any of those frontier directions could be the next hot thing.

So an illicit organization is going to operate a frontier scale data center and do $1b training pulling power from the grid without detection?

They very likely will via shell companies in jurisdictions outside the west. Are you suggesting we go to war to stop them? The massive datacenters are needed to serve models to millions. Criminal orgs don't need massive data centers anyway.

As Bruce Schneier recently discussed, law and tax law are code, just like source code. LLMs are great at finding holes in them. Illicit organizations looking to launder funds are most certainly interested in what AI can do for them.

It’s not a binary choice. There are more options than let the labs run wild and capture the full stack and then the whole knowledge economy or make them entirely illegal and move all activity to the black market. We can regulate to limit their ability to operate to just providing utility inference: make them divest codex, Claude code and any apps; they can only be an API that others build on. Limit their ability to buy compute to a specific amount of available supply so other companies are able to buy compute. Limit their ability to accumulate private training data and require that they make their training data publicly available for others to use after a certain period of time. Make them legally obligated to publish their weights so others can. All of this would increase competition and ensure that they don’t establish dominance over society.

Then we could require comprehensive logging of every tool call, model trace, chain of reasoning, and even tensor propagation all of which would be spot inspected like the CFTC does with commodity trading and settlement. We could have embedded auditors with specific risk analysis metrics like large banks do. We could limit tool calls to dedicated sandbox’s with a blanket prohibition on AI accessing user space. We could create a parallel internet for agents so they are only able to access Secure Enclave. Even if these measures aren’t 100% perfect they would reduce the risk.

Reducing which risk? The risk that the models of big American companies cause harm? What if they abide by some rules while models of others remain unfettered? That also carries risk. I appreciate the wish to have tame AI which only does good things but I'm afraid we're already beyond that.

I would frame it as inverse. Rather than aiming at reducing the risk of harm, our Ami should be matching the incentive to get personally rich with a corresponding risk of personal bankruptcy or criminal liability. That wouldn’t just incentivize safety, it would incentivize abstaining from activities with insufficient confidence of avoiding even long tail risk.

More importantly it would incentivize the entire industry at the same time by injecting caution into the yolo speed run we have now. No frontier lab or single user hobbyist would take imprudent risk in the hope that they get rich if they also risk the full weight of consequences.

No, why would that be the consequences of regulation?

Humanity has discovered ways to ensure that we don’t boil the planet, feed everyone, and get better healthcare to the people in the US.

We are very capable of putting good things in a box. We are just incapable of putting profitable things in a box.

Good examples. We are currently failing at all of that.

You're making a straw man there.

Nobody (weirdly) proposes to forget about nuclear weapons, doesn't mean everybody should have one.

When you dream about flying a dragon to work, reality poses e.g. parking issues and insurance mismatch as obstructions. Maybe settle for a bike instead?

Nuclear weapons take a bit more work than doing math.

Plus, looks like everybody is getting one anywayz

So does manufacturing GPUs.

So you want to regulate the sale of GPUs? How are you going to do that and why would it be more effective than currently failing measures to regulate nearly everything else?

I'm not advocating anything, to be clear.

Reply on news.ycombinator.com