SB-1047 will stifle open-source AI and decrease safety
answer.ai
answer.ai
This matches my thoughts on why this is ultimately a bad piece of legislation. It is virtually impossible to ensure that a piece of technology will not be used for "harmful purposes". I agree that such stipulations will be just another roadblock keeping everyone except "big businesses with well funded legal teams" from working on LLMs.
Trigger happy regulation for a field that hasn't even come into full swing. It's indicative of an over-active immune system; lawmakers with nothing better to do.
Pass laws against improper use and go after the malicious users. Don't ban the technology, the research, or even the applications. (Of which there will be abundant good uses. Many of which we've yet to even see or predict.)
Our culture has become obsessed with regulating and limiting freedom on the very principle that it might be harmful. We should be punishing actual measurable, physical and monetary harms. Not imaginary or hypothetical ones.
If California passes this, AI companies should leave California behind.
Where is the ban here?
> Other relief as the court deems appropriate, including monetary damages damages, including punitive damages, to persons aggrieved aggrieved, punitive damages, and an order for the full shutdown of a covered model.
> A civil penalty in an amount not exceeding 10 percent of the cost, excluding labor cost, to develop the covered model for a first violation and in an amount not exceeding 30 percent of the cost, excluding labor cost, to develop the covered model for any subsequent violation.
[1] https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...
A better-fitting analogy I'd make is that sex causes disease and other negative externalities, so we should pass laws that force people to be married and licensed in order to have sex.
In any case, this bill is the walking epitome of something a "nanny state" might produce.
[1] TikTok and Instagram have far more impact on this, and we've yet to do anything there. We seem to be of the opinion that this should be an individual responsibility.
Some of us worry that billions of people will be killed by AI in the future -- possibly without anything that you or the average decision-maker might regard as a warning. (They're likely to be killed all at the same time.)
I.e., it is more like a large asteroid slamming into the Earth than a stream of deaths over time such as produced by the deployment in society of the automobile (except that the asteroid does not have the capability of noticing that it's first plan failed to kill a group of human over there, then to devise a second plan for killing those).
Science fiction.
What really gets me concerned is the quality of the writing on the subject of how can we design an AI so that it will not want to hurt us (just as we design bridges so that we know from first principles they won't fall down). Most leaders of AI labs have by now written about the topic, but the writings are shockingly bad: everyone has some explanation as to why the AI will turn out to be safe, but there are dozens of orthogonal explanations, some very simplistic, none of which I want to bet my life on or the lives of my younger relatives.
Those who do write well about the topic, particularly Eliezer Yudkowsky and Nate Soares of the Machine Intelligence Research Institute, say that it is probably not currently within the capabilities of any living human or group of humans to design an AI to be safe (to humans) the way we design bridges to be safe, and that our best hope is the hope that over the next centuries humankind will become cognitively capable enough to do and that in the meantime people stop trying to create AIs that might turn out to be dangerously capable -- which (because outside of actually doing the training run, we have no way of predicting the effects on capability of the next architectural improvement or the next increase in computing resources devoted to training) basically means stopping all AI research now worldwide and for good measure stopping progress in GPU technology.
Eliezer has been full-time employed for over 20 years to work on the issue (and Nate has been for about 15 years) and they've had enough funding to employ at least a dozen researchers and researcher-apprentices over that time to bounce ideas off of in the office.
We were similarly ignorant about recombinant DNA, so Asilomar was very cautious about it. Now we know more, we are less cautious. I still think it was good to be cautious and not to dismiss recombinant DNA concerns as "science fiction".
The relevance is IMHO this bill is largely an ossification at the government level of the safety and alignment philosophy of the big corps. I'm guessing they mainly wrote this bill. It's not the specific words "safey and alignment" that matter, it's the philosophy.
If the bill were only covering AI killing machines I'd (probably) be in agreement with it, but it seems significantly more overreaching than that.
No harm done!
>If the bill were only covering AI killing machines I'd (probably) be in agreement with it, but it seems significantly more overreaching than that.
Just to make sure we are on the same page: my main worry is the projects ("deployments"?) that aren't intended to kill anybody, but one of those project ends up killing billions of people anyways. It probably kills absolutely everyone. That one project might be trying to cure cancer.
The only way of not incurring this risk of extinction (and of mass death) that I know of is to shut down all AI research now, which I'm guessing you would consider "overreaching".
It would be great if there were a way to derive the profound benefits of continuing to do AI research without incurring the extinction risk. If you think you have a way to do that, please let me know. If I agree that your approach is promising, I'll drop everything to make sure you get a high-paying job to develop your approach. There are lots of people who would do that (and lots of high-net-worth people and organizations who would pay you the money).
The Machine Intelligence Research Institute for example has a lot of money that was donated to them by cryptocurrency entrepreneurs that they've been holding on to year after year because they cannot think of any good ways to spend it to reduce extinction risk. They'd be eager to give money to anyone that can convince them that they have an approach with even a 1% probability of success.
I guess I need to decide how high I feel the risk is of that, and that I'm less sure of. Appreciate the discussion btw!
I'm not against legislation regulating AI, but it needs to be targeted toward clear problems e.g.: stealing copyrighted material, profiling crime, face recognition, self driving vehicles, automated "targeting" however you want to interpret that.
I want to point out above are some awful uses of AI that are leveraged mostly by closed, proprietary entities
I think they are having to deal with things like sales to countries outside of their legal reach. So, while I understand the tack here, there's probably more to it than this.
Of little concern in the US legal system. Might be problematic in the EU perhaps, but in the United States the courts have consistently been tremendously deferential to the interests of small and large businesses vs consumers.
I guess they are damned if they do and damned if they don't.
We constantly complain about slow lawmaking, "Look at how out of touch Congress are! XYZ technology is moving so fast, and they're always 10-20 years behind!" Finally, someone is actually on the ball and up-to-date with a current technology, and now the other complainers complain that they're jumping the gun and regulating too soon. Lawmakers can't win.
I don't think I've ever complained about slow lawmaking.
I've complained about a lack of rights and a lack of enshrined rights.
In Engineering, "best practices" are the set of "just do X" answers that will let you skip deriving every answer about what material or design to use from first principles for cases where there's a known dominant solution. For example, "for a load-bearing pillar, use steel-reinforced concrete, in a cylindrical shape, with a cross-sectional diameter following formula XYZ given the number of storeys of the building." You can (and eventually must!) still do a load simulation for the building, to see that the pillar can hold things up without cracking — but you don't have to model the building when selecting what material to use; and you don't have to randomly fiddle with the shape or diameter of the pillar until the load holds. You can slap a pillar into the design and be able to predict that it'll hold the load (while not being overly costly in material use!), because "best practices."
Otherwise you also have to complain about the stifling of open source bioagent research, open source nuclear warheads, open source human cloning protocols
Those are also all dual-use technologies that are objectively morally neutral
nuclear warheads?
Some outcomes are pretty terrible, I think there are valid instances where we might also want to prevent precursor technology from being widely disseminated to prevent them.
Circumvention also absent the DMCA isn't illegal.
They have to be about both because outcomes aren’t predictable, and whether something is an intermediate or ultimate outcome isn’t always clear. We have a law requiring indicator use on lane change, not just hitting someone while lane changing, for example.
The equivalent would be if the law simply said, "don't change lanes unsafely" but didn't define it much beyond that, and left it to law enforcement and judges to decide, so anytime someone changed lanes "unsafely" there's now extremely unknown legal risk.
This is directly analogous to requiring disclosures and certifications be filed with the state. Those are actions as much as hitting an indicator.
I haven’t read the proposed bill closely. But it seems to be a standard rulemaking bill.
No, actually you don’t.
This is just a slippery slope that suggests that any of these examples are even remotely comparable to AI. There is room for nuance and it’s easy to spot the outlier among bioagent research, nuclear warheads, human cloning, and generative artificial intelligence.
I hope I'm wrong
Maybe wait until you're sure before holding guns to peoples' heads.
(admittedly, I'm getting a bit motte-bailey here, but still)
It’s unhelpful to the argument when you do this, and it makes our side look like a bunch of smug self entitled assholes.
The reality is that AI is disruptive but we don’t know how disruptive.
The parent post is clearly hyperbole; but let’s push back on what is clearly nonsense (ie. AI being more dangerous than nuclear weapons) in a logical manner hm?
Understanding AI is not the issue here; the issue so that no one knows how disruptive it will eventually be; not me, not you, not them.
People are playing the risk mitigation game; but the point is that if you play it too hard you end up as a ludite in a cave with no lights because something might be dangerous about “electricity”.
The running away from responsibility is one of the things I like least about big tech.
LLMs are functions of their training data, nothing more. This is evidenced by how we see very different model architectures produce essentially the same result. All of that training data is out there, on the internet, in books; none of that “dangerous” knowledge is banned or regulated, nor should it be.
When a person uses a car to drive into a crowd, do we blame the automobile manufacturer? Do you blame Kali Linux when someone uses it to hack a remote system? What about Apple when an iPhone is used to call in a threat to a school?
So no thank you, bring back responsibility to end users of products, and allow suppliers to develop the best capabilities they can.
By all means, let's have responsibility for actual outcomes. That bill is talking about imagined outcomes.
(A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties. (B) At least five hundred million dollars ($500,000,000) of damage through cyberattacks on critical infrastructure via a single incident or multiple related incidents. (C) At least five hundred million dollars ($500,000,000) of damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human. (D) Other threats to public safety and security that are of comparable severity to the harms described in paragraphs (A) to (C), inclusive.
That means AI for drug discovery and materials science development, AI for managing electricity grids and broadband traffic, AI in the financial and health services sectors, etc. Then there's the military-industrial side, which this legislation might not even touch if only federal contracts are involved. Classified military AI development seems reckless, hasn't anyone seen War Games?
https://technologymagazine.com/top10/top-10-military-technol...
At least with open source, the capabilities are more immediately visible.
2. Folks aren’t automatically liable if their highly capable model is used to do bad things, even catastrophic things. The question is whether they took reasonable measures to prevent that. This bill could have used strict liability, where developers would be liable for catastrophic harms regardless of fault, but that's not what the bill does.
3. Overall it seems pretty reasonable that if your model can cause catastrophic harms (which is not true of current models, but maybe true of future models), then you shouldn’t be releasing models in a way that can predictably allow folks to cause those catastrophic harms.
If people want a detailed write up of what the bill does, I recommend this thorough writeup by Zvi. In my opinion this is a pretty narrow proposal focused at the most severe risks (much more narrow than, e.g., the EU AI act). https://thezvi.substack.com/p/on-the-proposed-california-sb-...
A regular MacBook can cause half a million dollars of damage if misused. Easily. So I think any model of significant size would qualify.
Furthermore, the requirement to register and pre-clear models will surely precede open data access, and that means a loss in competitive cover for startups working on new projects. I can easily see disclosure sites being monitored constantly for each new AI development, rendering startups unable to build against larger players in private.
Or even a piece of chipped flint or a pointy stick, if it comes to that.
"Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose a mortal threat to open societies.”
https://www.wired.com/story/mortal-danger-chinas-push-into-a...
Literally everyone out there who pursues global influence is just frothing at the mouth over AI. This is seriously tempting me to buy the 512gb Mac Studio when it comes out so I can run the big llama3 model, which will probably be banned any day now.
I hope someone has / can do some investigative journalism to check out their links with commercial / closed source AI; I can imagine the investors and those that benefit from companies like "Open"AI have close links with politicians. There's probably no direct links, they've become really good at obscuring those and plausible deniability.
Microsoft, Amazon, OpenAI, others are driving regulatory capture behind the scenes. The usual suspects are dropping all sorts of money on establishing control and rent seeking - actual open source AI with end user control makes it much harder for these asshats to extract money and exert influence over people, and they desperately want both. AI, like search, will be a powerful influence vector for politics and marketing.
We’re still in the very early days.
Unless the academic community really drops the ball, in 5 or so years they’ll be training models around the quality of the current state of the art on professors’ research clusters (probably not just at R1 universities).
I’d be shocked if, in the long term, anyone who can get access a library’s worth of text won’t be able to put together a useable model.
There’s nothing magical about our brains, so I imagine at some point you’ll be able to teach a computer to read and write with about as many books as it takes to teach a human. I mean maybe they’ll be, like, 10x as dumb as us. A typical American might read hundreds of books over the course of their life, what are they going to do, require a license to own more than a couple thousand e-books?
The long term plan for any lawmaker is winning the next election. Anything further in the future doesn't matter much.
The long term plan for incumbents here might be building a large moat by regulatory capture.
Maybe incumbents are helping lawmakers. Do ut des.
My guess: everything that slows down ai development is good, because it gives society time to adapt.
(I think this plan is flawed, because it is easier to adapt to open research than to closed research)
Probably only if you count books like green eggs and ham.
Wait, you're saying that a bunch of legislators who believe the Earth is 6000 years old may not have a valid perspective on complex technical matters? No. Say it isn't so.
On the other hand, like existing ITAR, this will manifest in extremely weird rules that have very little to do with actual safety.
>On February 7, 2024, Senator Scott Wiener introduced Senate Bill 1047 (SB-1047) – known as the Known as the Safe and Secure Innovation for Frontier Artificial Intelligence Systems Act (the Act) – into the California State Legislature. Aiming to regulate the development and use of advanced artificial intelligence (AI) models, the Act mandates developers to make certain safety determinations before training AI models, comply with various safety requirements, and report AI safety incidents. It further establishes the Frontier Model Division within the Department of Technology for oversight of these AI models and introduces civil penalties for violations of the Act.
https://www.dlapiper.com/en/insights/publications/2024/02/ca...
https://www.opensecrets.org/orgs/openai/summary?id=D00008425...
You've got a company valued at $80 billion and you can get legislation put forward to kneecap your primary competition for a the price of a 2009 Honda Accord with 150,000 miles on the odometer? What great value for money!
(pace Crazy Eddie)
This typically works the other way. You find politicians who support you, due to personal views or electoral idiosyncrasies, and then give them money to boost them.
https://www.sfgate.com/politics/article/fentanyl-dealers-in-...
Has the tech industry always been so cutthroat, or is this a new trend? Maybe I just didn't hear about tech companies lobbying for power in the past?
Here's a link[0] with 2023 lobby spends by industry, but there is no "tech" listing specific. There's an entry for "Internet" listed, which I'm guessing is what you mean by "tech". Another chart[1] breaks down that entry.
If you want to know when each company started to spend money, you could research their public filings.
[0] https://www.statista.com/statistics/257364/top-lobbying-indu...
[1] https://www.statista.com/statistics/1035987/us-leading-inter...
— Posted from my Xerox
The roots of the tech sector are PC (the libertarian dream of basically zero-cost startups), telecoms (playground of monopolies and regulatory capture), and ad guys who’s main trick is outrunning society’s ability to understand their business model.
Some nuggets...
---
So we are only allowed to train what they allow us:
This bill would require that a developer, before initiating training of a nonderivative covered model, comply with various requirements, including implementing the capability to promptly enact a full shutdown of the covered model until that covered model is the subject of a limited duty exemption.
---
Of course it comes with a new department with powers to impose fees:
This bill would also create the Frontier Model Division within the Department of Technology and would require the division to, among other things, review annual certification reports from developers received pursuant to these provisions and publicly release summarized findings based on those reports. The bill would authorize the division to assess related fees and would require deposit of the fees into the Frontier Model Division Programs Fund, which the bill would create.
---
And, obviously, we must pay consultants:
This bill would also require the Department of Technology to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale artificial intelligence models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform.
So you need to be able to turn it off. It should be easy? Still it seems to be a good thing to make sure you can turn it off.
The second half of that sentence clarifies. It's not that you have to be able to shut down the model, everyone knows it's trivial to turn off a computer. It's that the government can force you to shut down your program until such time as they give you regulatory approval to turn it back on again.
Doesn't this inherently break normal distribution of source code which inherently doesn't come with an off switch?
The gig worker min hourly pay is fine. If it decreases the total demand for deliveries thats ok. I wouldn't want more McJobs for the state to subsidize anyway. Those workers don't cease to exist they just work somewhere else for someone who can actually afford to pay.
The gig worker min rate has completely cut out their money, you can hear feedback directly from the gig workers and see that it's being reversed because of the backlash: https://www.newsweek.com/20-minimum-wage-law-seattle-deliver... -> "300,000 fewer orders within Seattle". I can't agree with you here at all. These are jobs people have the choice to take or not, the government here is eliminating that choice by basically making the jobs nonexistent. I know I've cut my orders significantly and will walk or drive myself nowadays to pick up food when I do get takeout.
"Those workers don't cease to exist they just work somewhere else for someone who can actually afford to pay" <- citation needed. Setting wage floors almost always get modeled out as shortages where supply of workers will no longer meet demand for jobs, and most people aren't perfectly fungible, nor are there are bunch of jobs that allow people to work for a few hours between other gigs, watching their kids, trying to be entrepreneurial, etc. Let adults decide which jobs they want to work for which pay. If there were alternative jobs that paid more don't you think they'd naturally flow there rather then the government stepping in?
The government disallows work that people would otherwise do all the time for instance by instituting minimum wage, requiring benefits, or requiring regulation that drive up costs enough that marginal businesses fold. Those employees don't cease to exist they are reallocated to other parts of the market which are more worthy. In the end uber eats isn't worth anything it loses money. It's a side show until investors money runs out.
There are parts of Seattle that are shady. Unfortunately those people exist and they aren't going anywhere so we are basically playing whack a mole. If we want them out of people's faces we should probably house them. Finland did and it worked for them.
https://thezvi.substack.com/p/on-the-proposed-california-sb-...
> Answer.AI is a new kind of AI R&D lab which creates practical end-user products based on foundational research breakthroughs.
It's very likely the company / author is dogfooding.
>Before initiating the commercial, public, or widespread use of a covered model that is not subject to a positive safety determination, limited duty exemption, a developer of the nonderivative version of the covered model shall do all of the following:
>(1) Implement reasonable safeguards and requirements to do all of the following:
>(B) Prevent an individual from being able to use the model to create a derivative model that was used to cause a critical harm.
This is simply impossible. If you give me model weights, I can surely fine-tune them into doing a covered harm (e.g. provide instructions for the creation of chemical or biological weapons). This requirement is unsatisfiable, and you're not allowed to release a covered model without satisfying it.
> The definition of covered model seems to me to be clearly intended to apply only to models that are effectively at the frontier of model capabilities.
> Let’s look again at the exact definition:
> (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024, or a model that could reasonably be expected to have similar performance on benchmarks commonly used to quantify the performance of state-of-the-art foundation models, as determined by industry best practices and relevant standard setting organizations.
> (2) The artificial intelligence model has capability below the relevant threshold on a specific benchmark but is of otherwise similar general capability.
> That seems clear as day on what it means, and what it means is this:
> 1.
> If your model is over 10^26 we assume it counts.
> 2.
> If it isn’t, but it is as good as state-of-the-art current models, it counts.
> 3.
> Being ‘as good as’ is a general capability thing, not hitting specific benchmarks.
> Under this definition, if no one was actively gaming benchmarks, at most three existing models would plausibly qualify for this definition: GPT-4, Gemini Ultra and Claude. I am not even sure about Claude.
> If the open source models are gaming the benchmarks so much that they end up looking like a handful of them are matching GPT-4 on benchmarks, then what can I say, maybe stop gaming the benchmarks?
> Or point out quite reasonably that the real benchmark is user preference, and in those terms, you suck, so it is fine. Either way.
> But notice that this isn’t what the bill does. The bill applies to large models and to any models that reach the same performance regardless of the compute budget required to make them. This means that the bill applies to startups as well as large corporations.
> Um, no, because the open model weights models do not remotely reach the performance level of OpenAI?
> Maybe some will in the future.
> But this very clearly does not ‘ban all open source.’ There are zero existing open model weights models that this bans.
So no, it does not seem that anything was missed.
(1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations.
(2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models.
---
f.1: anyone who has taken a basic cpu arch class knows that int and float are significantly different computational effort. One could see an entity using this in court to greatly lower the threshold for qualification after the fact. i.e. lawyers play word/text games and one could say something to the effect that 1 float is 10 int ops, so the limit is 10^26 int or 10^25 float ops
f.2: future proofing against better algorithms based on today's benchmarks... to the point where effort no longer matters. They seem to be drawing the threshold at today's benchmarks, whether or not they are reflective of capability. I could see a small model be trained to do poorly on these benchmarks while excelling at the problems they are concerned with, like making nuclear weapons...
> 22603. (a) Before initiating training of a covered model that is not a derivative model, a developer of that covered model may determine whether ... if the covered model will have lower performance on all benchmarks
because I know how it will perform before training?
It essentially creates 2 thresholds
1. number of math operations
2. 2024 benchmark results
100% agreement this will hurt CA and others, especially nefarious, will ignore it
Your personal opinion as to what level of AI tooling was used in crafting the content is a low value signal and posting that speculation without meaningfully engaging in the content is as low value as commenting to say "this article sucks".
Personally, I think this would be a very strange place to find pute AI genetated content. This is a personal statement that was submitted to the state, and posted under a real name under a site the poster has a professional association with. I think that any "strangeness" in formating a wording comes from the role this text serves, as a public comment intended to affect policy.
I really like this proposed model. Are there good arguments against this?
Llama3-400B might be well past this threshold.
[1] https://ai.meta.com/blog/meta-llama-3/
[2] https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct
Another point that gets buried is that AI is about the data. Without the transparency of what these models are built from, it leads to potential dangers as well as inappropriate use of materials.
My bet is on the open ecosystem if it doesn’t get legislated away.
Proponents of the regulation approach probably wouldn't state it this way, but if I'm understanding their arguments correctly, I think they want that, because regularing a few very powerful corporations is easy. Regulating a ton of small people/startups is hard. When you genuinely believe that some of the output from LLMs is literally dangerous to some people, it's not unreasonable to decide that the "freedom" of people to run and develop models to compete is unimportant compared to protecting society from dangerous text or images.
This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.
I've seen that number thrown around, and by some calculations it would already apply to open source models like LLama.
No, we don't need to ban or regulate LLama or any existing open source models. If someone wants to be worried about GPT-6, fine. But there is no need to regulate the stuff thats already out there.
It's dumb to create laws based on arbitrary technological limits. Computing power is still increasing exponentially, yesterday's supercomputer is tomorrows gaming gpu.
[1] https://ai.meta.com/blog/meta-llama-3/
[2] https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct
Thomas Watson, president of IBM, 1943
1. The new Frontier Model Division is focused on receiving information and issuing guidelines. It’s not a licensing regime and isn’t investigating developers.
2. Folks aren’t automatically liable if their highly capable model is used to do bad things, even catastrophic things. The question is whether they took reasonable measures to prevent that. This bill could have used strict liability, where developers would be liable for catastrophic harms regardless of fault, but that's not what the bill does.
3. The bill requires developers to test their models and report whether they have hazardous capabilities (and the answer can obviously be yes or no). Even if the model does have hazardous capabilities, the developer can still deploy it if they take reasonable precautions, as outlined in the bill. For perjury, you would need to intentionally lie—good faith errors would not be covered. I get that models can have unforeseen capabilities, but this isn’t about that. If you are knowingly releasing something that could have demonstrably catastrophic consequences, it seems fair to have consequences for that. Some things which already require folks to certify under penalty of perjury: lobbying disclosures, companies’ financial disclosures, immigration compliance forms.
4. Overall it seems pretty reasonable that if your model can cause catastrophic harms (which is not true of current models, but maybe true of future models), then you shouldn’t be releasing models in a way that can predictably allow folks to cause those catastrophic harms.
If people want a writeup of what the bill does I recommend this one by the law firm DLA Piper (https://www.dlapiper.com/en/insights/publications/2024/02/ca...). In my opinion this is a pretty narrow proposal focused at the most severe risks (much more narrow than, e.g., the EU AI act).
Legislation should not restrict the development or operation of fundamental AI technologies. Instead laws should only be built on the specific uses that are deemed illegal, irrespective of AI.
> Regulate the use of AI in high-risk areas such as healthcare, criminal justice, and critical infrastructure, where the potential for harm is greatest
This suggests, for example, image generation should be unregulated, but potential for harm of deepfake is great. In general, regulation needs to be feasible to implement, and even if it is ideal to regulate use not development, it can make a sense to regulate development due to feasibility concerns.
It wouldn't be inadvertent. It's a control tactic.
"Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams." ... "These requirements could disproportionately impact open-source developers who often lack the resources of larger corporations to navigate complex regulatory processes." ... "The proposed regulations create significant barriers to entry for small businesses and startups looking to innovate in the AI space."
That's the idea. The government likes a small number of big businesses that they can control.
Then why post on answer.ai?
from the bill, the specific applications to open source are:
Appoint and consult with an advisory committee for open-source artificial intelligence that shall do all of the following:
(A) Issue guidelines for model evaluation for use by developers of open-source artificial intelligence models that do not have hazardous capabilities.
(B) Advise the Frontier Model Division on the creation and feasibility of incentives, including tax credits, that could be provided to developers of open-source artificial intelligence models that are not covered models.
(C) Advise the Frontier Model Division on future policies and legislation impacting open-source artificial intelligence development.
nowhere does it state open source developers need "required shutdowns" or burdensome reporting for open source. The states position to regulate trade is sacrosanct and in such, the bill applies almost entirely to commercial products. it would affect Jeremys business and as a business owner, he doesnt like that.
here is the actual bill
Any reporting to a gov agency is burdensome. It all exists to stifle private development.
No, just no. Have you ever been to a University?
One might question why that is. Perhaps it's the case that Jeremy has an excellent response to these points which he has somehow neglected to raise. Or perhaps it's because these threats are very inconvenient for an open source developer.
I'm sure he'd say that open-sourcing models means that all actors have access to defensive systems and that the good guys outnumber the bad guys and it'll all work out well.
And that could be true. Or it could be false. It's not like we really know that everything would work out fine. It's not that we've run the experiment. I mean maybe it works out like that, or maybe one guy creates a virus and then it doesn't really matter how many folk on the other side, but we still get kind of screwed because we can only produce vaccines that fast. It's that's what going to happen? I don't really know, but it's at least plausible. I mean, maybe we'll automate all aspects of vaccine production and be able to respond much faster, but that's dependent on when we develop this technology vs. when AI starts significantly helping with bioweapons with someone then using it for an attack. And at that point it's all so uncertain and up in the air that it's seems rather strange for someone to suggest that it'll all be fine.
LLMs are not intelligent, they predict text based on what it was trained, if it could somehow build new viruses, weapons then it means the internet has MANY such information so the LLC could predict something useful, so maybe those websites, scientific papers , blog posts need to be deleted because some extremist group or state sponsored group can use them directly plus Natural Intelligence plus good laboratories.
But tell me how can I make my next LLM so it would help on say fighting biologic weapons, creating vaccines but refusing to make evil stuff keeping in mind that jailbreaking is always possible (scientifically proven)
Do you know how much skill, practice, resourcing and time it takes to develop bio-anything?
Instead of writing your senator who is owned by OpenAI - just throw away your `OPENAI_API_KEY` and use one of the many open models like mistral or llama3.
ollama run mistral
And there are cloud providers like https://replicate.com/ and https://lightning.ai/ that will let you use your LLM via an API key just like you did with OpenAI if you need that.
You don't need OpenAI - nobody does.