Hacker News Reader

Comment by myrmidon | Hacker News Reader

Again, you are assuming that frontier models will stay meaningfully ahead long-term. Export controls/bans are pointless if this is not the case.

There is ton of strong indicators that they will not stay ahead: Assuming that technological progress of any kind follows some form of logistic function (where "gains", in this case "intelligence" become sub-linear at some point) is (long-term) a very conservative and proven assumption, and "automatically" negates your lead over time.

Similarly, purely "intellectual" advantage in disciplines like cryptography/computer chess/algorithm design never really stayed concentrated, either.

View on HN

I would take the other side of this bet. While I agree that the impact of any given advance is likely to resemble a sigmoid curve, I think there is a material chance of "stacking sigmoids" creating something that looks exponential.

To take a simple example, look at the progress of technology over the last ~500 years - it seems to me that the rate of change continues to accelerate despite many of the logistic curves flattening along the way.

There are still huge unanswered questions about whether or not the stacking sigmoids will favor the incumbents. But I would not definitively bet against the people with the most compute data, talent and money.

I'm unsure what exactly you mean by sigmoid stacking.

I'd argue that a lot of important technologies (like circuit design!) started at basically zero in the last century, and the progress was actually exponential for a significant time (=> because we started so low on the curve).

But if you reduce things to a single metric (wealth per person? total energy available to humanity? global industrial/construction output in tons?) I can't think of anything where such "stacking" successful subverted the sigmoid trend (or looks like it will, long-term).

It is not about intellectual advantage, but about capabilities as well. Assuming that Europe would get all the knowledge they need on the cutting edge technology on how to train a frontier model (which they won't because US and China would guard this as national secrets), who is going to setup infrastructure to train the model? Who is building the data centers (multi-year project) and who is going to build the energy infrastructure? Just because you know how to do it doesn't mean you have the capability to do that as well.

One of the best example is nuclear reactors. By now the know how and technology is fairly mature and open, but not every country gets to build nuclear reactors. Same would be with the frontier models as well.

EU should have already started investing in the infrastructure side in-case they obtain the know how, but your politicians are still bickering on pension reforms and Ukrainian war, etc.

So Deepmind/Meta/etc... would all have to cut off their EU offices even though they hold incredible talent? I'm just not finding this scenario likely.

Does meta has an AI training data center in EU?

They can just operate and provide normal access to their services, just block AI access. This is already happening, apple would not release new Siri in EU (granted it's due to a regulation clash) but this would be a testing point.

If Europeans are still paying the same prices for sub par services/products to their American counter parts, it's win-win situation for those companies. Just sell a dumbed down non-AI version of service/product in EU for an inflated price.

Meta's FAIR has several R&D offices in the EU, yes. So you are saying their labs can conduct R&D on models in the EU, potentially even train them there, they just can't have production LLM inference serving or release the model weights? I'm just not seeing it.

A not-insignificant portion of the AI/ML research community is in the EU.

What you’re failing to understand is that meta is ultimately a US company. If meta unveils a powerful frontier model tomorrow and US impose a ban of its exports, regardless of how many research, data centres, meta has in EU, they will restrict access to the model for EU citizens, same way Anthropic did.

Regarding the open weights, I don’t see meta doing that for their future models, especially once they have their own frontier models. Open weight models are kinda marketing strategy where they use it as a bait and switch. A lot of Chinese companies became popular with their open wight models and once they build that reputation they have no incentive to keep on releasing new open weighted models

meta researchers from eu will create new company and build model using knowledge they have.

In an imaginary world yes. Just because they know how, doesn’t mean that they can. It’s not like you can code a new model, you need compute power to train it, which they don’t have in EU.

Also meta could just take the best mind in EU research centres to US with millions of dollars as yearly payments

I agree is that money is an issue, but not geo-boundaries. If EU decides to commit XXXB they may have a chance to catch up eventually.

EU already has supercomputers and data centers. Some examples: https://www.eurohpc-ju.europa.eu/supercomputers/our-supercom...

The goal is to triple the number of data centers over the next years. It is not like there is nothing going on...

Again, super computers are not used to train frontier models. Please do your own research before blobbing just random links. By the time they triple those data centres the AI race is already lost

GB200 and GB300 are decent for training LLMs? H100 can be used also? Though the current European deployments are rather small, even the biggest are just some thousand GPUs.

Reply on news.ycombinator.com