Do Foundation Model Providers Comply with the EU AI Act? | Hacker News Reader

Do Foundation Model Providers Comply with the EU AI Act?

crfm.stanford.edu

Seems to EU is determined to cripple their AI industry at all costs, we already have so little technology companies...

Foundation models are labelled as 'high risk'!

In my opinion this is way too premature... this would cripple open source AI as well...

> While the act includes open source exceptions for traditional machine learning models, it expressly forbids safe-harbor provisions for open source generative systems.

>Any model made available in the EU, without first passing extensive, and expensive, licensing, would subject companies to massive fines of the greater of €20,000,000 or 4% of worldwide revenue. Opensource developers, and hosting services such as GitHub – as importers – would be liable for making unlicensed models available.

>Open Source LLMs Not Exempt: Open source foundational models are not exempt from the act. The programmers and distributors of the software have legal liability. For other forms of open source AI software, liability shifts to the group employing the software or bringing it to market. (pg 70).

Source: https://technomancers.ai/eu-ai-act-to-target-us-open-source-...

While i'm usually pro-EU they are really overreacting here and the consequences for our economy of crippling a technology with so much potential will be enormous in the long run.

That article is pure FUD.

ALL the regulatory activity mentioned in the article is related to "high-risk" AI systems, which are specifically:

- where the AI is part of a safety system, and where that safety system already needs to undergo conformity assessment.

- where the AI system poses a significant risk of harm to the health, safety or fundamental rights of natural persons - for very specific use-cases

https://www.europarl.europa.eu/resources/library/media/20230... (p122-125)

If this wouldn't apply to these foundation models then why would they write an article on how they currently comply or not?

> We assess the compliance of 10 foundation model providers—and their flagship foundation models—with 12 of the Act’s requirements for foundation models

The whole point of this article is to see what would apply to theses models!

Since there's some confusion about this:

- The AI Act regulates both 'high-risk AI systems' and 'foundation models' and applies different requirements for them.

- 'foundation models' are essentially defined in the act as "very large scale and expensive generative ai models that will probably only be offered via API" (my words). The reason the act wants to regulate them is so that USERs of foundation models have a chance to make their downstream use case complaint if that use case is high-risk. For example, if I'm a health insurance provider and I'm using a chatbot enabled by GPT4 in my health insurance sign-up flow, then my system may be high-risk and needs to be compliant. I need access to some information aobut GPT4 (e.g. expected error modes, potential biases etc) to do that.

- The wording of the act makes a point of highlighting that your run-off-the mill open source generative AI project will not constitute a 'foundation model'. The exact scale at which a project will become a regulated 'foundation model' is not yet clear, but it can be assumed that it will be at least tens of millions of dollars. If you can spend that much on compute an researchers, I think you can spend a few k on becoming compliant.

- The technomancers article confuses requirements for High-risk systems with those for foundation models. (It also gets some of the high-risk requirements completely wrong, but that's another discussion.)

- The stanford HAL website does a great job with the facts! I really value seeing thoughtful contributions to the discussion like theirs. (Especially from an American institution!)

>where the AI system poses a significant risk of harm to the health

We've seen pushes from both sides to redefine anything they disagree with as harmful to mental health.

Both sides of what?

The AI Act EXPLICITLY enumerates all use cases that will be considered 'high-risk' (Annex III). If your use case is not on the list (or on the 'prohibited' list), then you're good to go. There's no mechanism where someone opposed to your model can argue you should be high-risk because of supposed harms perceived or dreamt-up by some political group. (Caveat: The list of high-risk use cases will probably be able to be amended by the Commission unilaterally after the regulation is enacted.)

it goes further than that. the technomancers blogpost gets a lot of the actual requirements completely wrong (for example the supposed requirement for third-party or government "licensing". Which is nowhere in the Act).

What really frustrated me about this whole discussion is seeing some SV heavyweights quoting this article uncritically and screaming about how stupid the EU is again, while referring to supposed requirements that are nowhere to be found in the act. I would assume these people have access to the best information in the world, yet they don't seem to have had any of their staff actually read the draft. :/

FWIW, I quickly wrote up some of my thoughts about what the technomancer's article gets wrong at the time, but then didn't get around to polish and publish them. If you're interested, here are my notes: https://gist.github.com/heidekrueger/bdee0268ecdad5f6b56f557...

Edit: I want to emphasize that I DO share some of the concerns that the blogpost raises about the current draft of the act. I just wish we could have a meaningful discussion about it rather than namecalling and fearmongering.

In my opinion it's about time to clarify rules and standards for automated systems that can for example kill someone if their output is incorrect.

ChatGPT and stable diffusion are killing people now? I think incorrect output of regular software is actually killing people but we don't need a license to write code.

I chose my words to include this "regular software".

Do you plan to integrate ChatGPT or Stable Diffusion to a high-risk system? I would advise against it at this time.

Maybe we should, for high risk applications.

You think software is bad now? Imagine if the world got 1% as much practice writing software because it was a licensed profession.

Why is that a problem for software but not other licensed professions such as law and and *actual* engineers.

Well, for starters I don’t want to live in a world with better lawyers, and secondly, by “actual engineers” do you mean civil engineers? In the US anyway, most engineers are not licensed excepting some civil engineers.

EU crippling innovation through regulation is a know mem3 but can you actually name regulation, which crippled innovation?

It's usually the Americans who are freaking out over this, I assume based on their local experience. It’s the same thing about the unions or any other stuff that works completely differently in EU and USA. it cuts both ways, European understanding of the American healthcare system is also a caricature.

wrong analogies also help, like assuming that EU is like the American federal system or that the European law works like the American law.

In the specific case about artificial intelligence, EU is interested in regulating high risk systems, but the online conversation revolves around people freaking out that EU will ban their home grown language model.

I am from the EU, not an American. I am not anti regulation in general. An example of where this has happened before was with GMO's:

> In 2006, the World Trade Organization concluded that the EU moratorium, which had been in effect from 1999 to 2004,[12] had violated international trade rules.[13][14]

We had a moratorium for years and even now we have the most stringent GMO regulations in the world. This crippled GMO research in Europe.

> This crippled GMO research in Europe.

Did it?

But yes, if something is banned then its probably bad for business and innovation around it. Europe is probably missing out on carcinogenic food additive innovations too. Okay, GMO's are not banned by EU but they are banned by some of the largest EU countries and if those countries fell behind in GMO tech, maybe it's due to the local bans and not the EU regulations?

EU isn't into banning AI though. Regulating doesn't mean banning, consider EU's digital markets act - EU didn't ban AppStore but put a requirement on device manufacturers to allow other AppStores. When EU regulated the EV charging ports, it did not ban EVs, but put a requirement to have unified plug.

Ah, the industry is out in force spreading FUD thick and early.

Here's what EU AI Act is actually about: https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...

And no wonder the (mostly American) AI industry is so outraged by it: it requires accountability.

Tracking "Energy" for what is otherwise a compressed version of the entire internet at your disposal seems so green-washed and disingenuous. Whatever "Energy" consumption those models have - it's peanuts compared to the alternatives.

The energy/environmental costs may be small compared to the potential usefulness of foundation models, but it's hard to make that assessment without data.

It's still an useful metric, why not measure and report it? Especially for models that are being deployed on mobile phones or desktop apps. And looking ahead, the future where IoT devices will have a mini-neural-net is not inconceivable.

One of the lessons from the crypto fiasco is that, if unchecked, the energy requirements can baloon to stratospheric heights.

I do think this will be a useful metric, and it seems obvious that the hyperscalers will have a feature helping you keep track of energy use and emissions of the resources you rented. But why demand this on the level of an individual model/product? For these foundation models, I think it's reasonable to assume they will all be trained on hyperscaler-provided gpu-clusters, so there'll likely be an off-the-shelf funcitonality by AWS/Azure/GCP to report this number, but the draft of the EU AI Act also demands tracking energy use for other 'high-risk' AI systems which companies may plausibly train and/or deploy on-prem. Good luck tracking the per-token energy use of your model that's running on some on-prem server on last-gen GPUs.

Especially for a server GPU, looking up watts and multiplying by time per token should give you a pretty good number.

Sure... but maybe the GPU is sitting idle 40% of the time while still consuming 200W. Should I have to break this idle energy consumption down onto actual use (assuming the server/gpu is only used for this one model)? I guess it would make sense, but... WHO should do this and then continually update the model documentation when idle rates or the hardware changes?

Listing it per server design (with groups) makes sense to me.

It wouldn't make sense to include measured idle time in the energy numbers you'd include in model documentation. Maybe that could go in a monthly report somewhere, but that's a different topic.

The organizations that release the models already provide (brag about) their model performance. They could simply include in the same report the info about the energy spent doing the training/finetuning/inference, per X tokens.

This doesn't necessarily measure every use, just "manufacturer's spec", the same you get for eg energy class for house appliances (at least in the EU). Nobody goes around measuring refrigerator power usage, but when you're buying one, you get a rough indication of how "green" (or not) it is.

I agree, that seems reasonable!

I was referring more to the users of such a system (what the AI Act would call a 'deployer'). They may have significantly less expertise but could still be required to track real-time energy use. Of course, simply referring to the 'energy label' by the provider could be a viable solution.

„It’s useful, why not“ is not a good basis for regulation.

Although the EU seems to think otherwise time and time again.

> "It's still an useful metric, why not measure and report it?"

What is it actually useful for?

I had the chance to talk to a staffer of one of the MEPs leading the political negotiations in EU parlament committee a few weeks ago. His take was that the pro-tech/pro-business parties conceded the 'AI users must track their energy use'-point to the Greens in the latest draft (which is the parliament's counterproposal to earlier drafts by the commission --think EU executive-- and council --think governments of the member states--) because it's so unrealistic in practice that it's likely to be stricken out of the law again during the final negotiation round between parliament and council negotiators.

I really hope that'll be the case. FWIW, I believe companies _should_ be required to keep tabs on their (and their supply chain's) emissions, but demanding that this be done at model/system level by data scientists is just ridiculous.

edit: grammar

The problem is, as soon as it's a law, there will be some official way to calculate it, penalties for misreporting, perhaps even a professional who must audit the energy use. Etc. Getting that number gets expensive!

Whereas if it is a company voluntarily reporting it, the number would just be number of GPUs * wattage of GPUs / tokens generated past year = energy per token.

I agree and share this concern in principle.

But... have you seen the state of GDPR enforcement? Anyone who made an honest effort is fine. I don't know of any GDPR enforcement action where the indicted company wasn't blatantly and willfully violating or ignoring the law.

FWIW, everything I've seen from regulators and the legislators involved in the nitty-gritty of the act seems to suggest that most of them are really smart people who know what they don't know. They know that AI is quickly evolving and the draft of the law goes out of its way to _not_ be too specific about _how_ to comply. E.g., I would not expect the EU (or national regulators) to bring down 'one right way' to report energy consumption.

The fact that they COULD still bothers me.

I believe the concern is everyone wants a foundation model, each one may have a comparatively tiny energy/carbon cost (vs yearly global usage and carbon output) but if there's lots of organisations training them up this all adds up. The energy data can help decide how much of a problem this is to worry about (solved through open models you can fine tune rather than always starting from scratch or maybe some regulations around model licensing that looks like the FRAND terms you see for patents and other IP.

Pretty sure a google search is order/s of magnitude cheaper than a chat gpt one. Training costs may not be comparable. I’m not sure.

This is as if the EU required Coca Cola to print the recipe on the bottle. They would just stop selling original Coke in the EU rather than disclose that.

What is more likely is that OpenAI, Google etc will train EU-specific models for the EU market.

It's more like requiring Coca-Cola to print which ingredients they put into it.

...and telling them they cannot put real cocaine into it.

Their goal is probably just to get rid of European competitors and get an advantage. To make sure big enterprises will have time, money and resources to get trough the whole process of compliance.

Or just lobby the regulators like alcohol industry did.

Is there a term for a country where ...

    - there are so many laws and regulations,
      that everybody is violating some of them

    - therefore only a tiny fraction can be
      brought to court

    - therefore everybody lives in the constant
      fear that they suddenly arbitrarily get
      crushed

?

United States of America? I may be biased by reading mostly English-language content on-line, which predominantly focuses on US issues, but my impression was that this is exactly what's happening in the US due to volume of regulation at state and federal level, and I've seen plenty of articles and discussions bringing this up.

it's called "normal country". It turns out that there are so many laws that we are breaking the all the time.

Examples:

https://lawlinguists.com/break-law-every-day/ https://mccreadylaw.com/blog/breaking-law-every-day/ https://www.rd.com/list/weird-laws/ https://brittontime.com/2021/01/08/top-10-ways-people-breaki...

> The legal validity of training on this data as a matter of fair use, especially for data with specific licenses, and of reproducing this data, remains unclear.

It's noteworthy that Fair Use is a largely American doctrine that does not appear in European copyright law across the board.

I was actually fearing something very stupid coming from the EU, but the requirements/questions asked (if it's only that from the table) are fairly ok!

350 pages of ramblings that nobody who was voting for this act hasn't actually read?

In comparison, the Inflation Reduction Act, an one-off act, is 750 pages long.

“If you don't want them to know something, put it in a book, they'll never read it” Idea as an old as the world itself :)

You are one wicked free thinker dude.

I wish haha, they are just following the guidelines :)) https://www.openculture.com/2022/01/read-the-cias-simple-sab...

When you write code, do you also ignore edge cases to keep things simple?

Yeah but a two-page font size 20 alternative proposal would introduce a thousand obvious and another ten thousand non-obvious loopholes, much like that double negative of yours. (I'm not trying to be a grammar Nazi, English isn't my native lang either) One of the reasons why the Constitution of USA has to be interpreted constantly is because it's relatively short and vague.

There is this weird and very real similarity between code and law. I realised this when I had to read up on some laws that were relevant to me, and found them shockingly easy to understand despite never having studied law.

Yeah I found something similar as well. It's all logic.

Or of an actual sensible legislation that requires accountability from AI development and deployment: https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...

It's notable that Hugging Face's BLOOM (https://huggingface.co/bigscience/bloom) might already be compliant (ignoring the 'member states' requirement which I'm sure they could comply with easily enough, it's about disclosing EU member where the model is on the market, so them simply listing all EU countries in a doc somewhere may suffice).

Some may be tempted to point at the lower scores of OpenAI, Google and Facebook models and say this means the regulations aren't fit for purpose, though of course you can take the view this means OpenAI, Google and Facebook aren't doing a good enough job of how they train and publish data about their models.

The full marks for Google in the 'member-states' section made me laugh out loud. -- Bard is not available in the EU. ;)

Good point, so really it should have full marks across the board ;)

I gotta say I don't see the point of the two compute requirements - the measure energy and reduce energy consumption part seems like it would be something a company would want to do by itself as that is a cost.

and I'm not sure what disclosing the training time and power used to train the model tells anyone?

on edit: tells anyone that is a consumer of the product and might have legitimate worries which would be a good prompt to legislation, added edit in for eager HN downvoter who didn't like my asking the question.

I think they just learned something from the history of crypto currencies, and this time they want hard data from the beginning. I know it's really apples to oranges, but having this 10 years ago for crypto might have pushed the tech from proof-of-work to e.g. proof-of-stake from the beginning.

And in the end there's no limits in this act. It's not like they're saying "No models above 10 GWh inside the EU!".

Why should there be models above 10GWh inside the EU?

We are a Japan based AI firm. We will not comply with the EU AI Act. We will be releasing our AI models MIT open source. Let them deal with that.

The EU is terrified at this news.

I expect that the EU will have enough regulation to make EU native AI companies comparatively uncompetitive but not actually enforce those regulations because of the economic and political disadvantages of doing so. Seems like the worst of all worlds to me but what do I know.

Well, I'm not sure about the EU AI Act, but I'm pretty sure they're still struggling with the 'Don't Turn Skynet On' Act.

It is important to note acts of EU parliament are just one stage of EU lawmaking and the fact "an act has passed" doesn't make it a law. Once such act has been passed there are many steps EU as well as member states have to take to make it law.

So, this article is very useful, but it really is just an analysis of compliance with _proposed_ law.

For it to become real law EU council has to agree unanimously (made up of prime ministers of all member states). Then each one of 27 countries has to implement the law following it's democratic process (national parliament, president, it has to pass any constitutional challenges if any are made etc). Only then it becomes law.

It is a long tedious process and certain things are apriori excluded from EU's jurisdiction altogether such as state level energy generation, anything that affects security situation and many others. So for decades now EU commission and the Court of justice of EU have been working very hard on "scope creep" of existing laws. It's a bonanza of opportunity for most powerful EU states to squeeze the smaller ones, for powerful external groups to influence whatever they want and so on. Seriously, after the horrible fiasco of Brexit (for EU, as I'm considering it from that perspective), loosing one of the most developed and competitive countries on Earth the EU should really have had a proper reform. There are many reasons why EU couldn't retain UK as a member. Arrogance of the commission, is but one of them. Now from the perspective of many years it is very clear booting UK out (and making it think it was it's own idea) was Franco-German (Russian sponsored) plan from the start.

This is an Act, it won't be nationally implemented but is applicable directly in EU states.

"Acts" don't exist in EU law. You either have Regulations (which become law after being approved) and Directives (which member states are bound to implement in their own legislation). "EU Acts" are just the new BS PR way to add yet another level of abstraction to European politics.

There are legislative and non-legislative acts. For the "Acts" that indeed are law the EU council can oppose them and the parliament has no power to override it. Eu Parliament is basically an auxiliary body. This particular "act" is a proposal for a regulation. It's not law itself. If it was going to be EU council would have to be involved and it makes the final decision. EU parliament only "accepts legislative acts" if EU council agrees, also negative results of such vote can be disregarded. It's not a real parliament. It's a place to transfer tens of thousands of Euro per person per month to people that pretend to make law. Funnily enough, EU parliament is the only truly demand cratic body in the EU, but it has the least lawmaking capability.

There is lots of text on sources like "Wikipedia" that make it sound like EU parliament is almost as powerful as national parliaments. This is simply untrue. First and foremost EU parliament hasn't even got the most basic parliamentary perogative of initiating proceedings in a legally binding act. Instead of Wikipedia I suggest this page (EU own) that shows various types of legal documents in EU. https://european-union.europa.eu/institutions-law-budget/law...

Of all these EU parliament can only make on its own "opinions" and "recommendations". Every other type gets initiated (the very first draft of text is written) by another body. Then the parliament votes on the proposal, but get this... The EU council is by no means bound by a result of that vote. So EU parliament can "strike down a proposal" and the EU council can still progress it. All it has to do is "take it under consideration". See a good factual description of the procedure here on a EU page https://commission.europa.eu/law/law-making-process/types-eu...

As for the act in this article it's not even an act that can become law in its current form. That particular act will now enter into a "draft negotiation mandate" phase when all real stakeholders (nation states) get to say if they like it, how they want to change it etc. And maybe then at some point it'll become law, in what capacity? With what wording? Who knows.

Reply on news.ycombinator.com