Hacker News Reader

Comment by simonw | Hacker News Reader

The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy

Something that caught my eye in the terms:

> Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma.

One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not thrilled by that particular clause.

View on HN

This is actually not that unusual. Stable Diffusion's license, CreativeML Open RAIL-M, has the exact same clause: "You shall undertake reasonable efforts to use the latest version of the Model."

Obviously updating the model is not very practical when you're using finetuned versions, and people still use old versions of Stable Diffusion. But it does make me fear the possibility that if they ever want to "revoke" everybody's license to use the model, all they have to do is just post a model update that's functionally useless for anything and go after anyone still using the old versions that actually do anything.

These are all very new licenses that deviate from OSI principles, I think it's fair to call them "unusual".

I think they meant not unusual in this space, not unusual in the sense of open source licensing.

For this sentence to parse, you need to either add or remove a "not".

That's useful context, thanks - I hadn't realized this clause was already out there for other models.

Switching to a model that is functionally useless doesn't seem to fall under "reasonable efforts" to me, but IANAL.

So if they wish to apply censorship they forgot, or suddenly discovered a reason for, they want you to be obligated to take it.

Good faith possibilities: Copyright liability requires retraining, or altering the underlying training set.

Gray area: "Safety" concerns where the model recommends criminal behavior (see uncensored GPT 4 evaluations).

Bad faith: Censorship or extra weighting added based on political agenda or for-pay skewing of results.

We are already culturally incapable of skillfully discussing censorship, "fake news", etc, this adds even more fuel to that fire.

It is an interesting time to be alive!

Sounds like it would be interesting to keep track of the model's responses to the same queries over time.

> Gemma-2024-Feb, what do you think of the situation in the South China Sea?

> > The situation in the South China Sea is complex and multi-faceted, involving a wide range of issues including political conflicts, economic challenges, social changes, and historical tensions.

> Gemma-2024-Oct, what do you think of the situation in the South China Sea?

> > Oceania has always been at war with EastAsia.

This is a great idea; I wonder if anyone is working on AI censorship monitoring at scale or at all. A secondary model could compare “censorship candidate” prompt results over time to classify how those results changed, and if those changes represent censorship or misinformation.

There's also (I think?) been some research in the direction of figuring out more abstract notions of how models perceive various 'concepts'. I'd be interested in the LLM version of diffs to see where changes have been implemented overall, too.

But really, the trouble is that it's tough to predict ahead of time what kinds of things are likely to be censored in the future; if I were motivated to track this, I'd just make sure to keep a copy of each version of the model in my personal archive for future testing with whatever prompts seem reasonable in the future.

Why the hell do they use such a crappy license in the first place?

I don't think a broken model would trigger that clause in a meaningful way, because then you simply can't update with reasonable effort. You would be obliged to try the new model in a test environment, and as soon as you notice it doesn't perform and making it perform would require unreasonable effort you can simply stay on the old version.

However you might be required to update if they do more subtle changes, like a new version that only speaks positively about Google and only negatively about Microsoft. Provided this doesn't have an obvious adverse impact on your use of the model.

It's worth noting that Stable Diffusion XL uses the OpenRAIL++-M License, which removed the update obligation.

I don't think there's a way they can enforce that reasonably. There's no connection to the mothership to report back what version is being used or license keys at runtime...

Seems more like a "if we discover something unsafe you should update your model and we aren't liable if you don't" than something that would make your model stop working.

This kind of defensive statements in ToS are usually due to obscure regulation or leading cases and model developers need a way to limit liability. There's no practical way to enforce this, but they can claim that when bad things happen it's purely on model users rather than model developers.

They have to make sure you’re receiving the most cutting edge chiding lectures when you make naughty and problematic requests.

You can't make a local model do that. eg force the answer to begin with "Yes" or use control vectors so it agrees with it.

This is strangely reminiscent of the Soviet Union, where after they got rid of Lavrentiy Beria, they mailed the update to subscribers of the Great Soviet Encyclopedia, where they asked to remove the three pages with Beria’s biography and replace them with the three provided pages.

Sounds like it's "reasonable" for you not to update then.

It says you must make efforts (to a reasonable extent), not that you must give a reason for not making efforts

This is a TOS, meaning their enforcement option is a lawsuit. In court, if you convincingly argue why it would take an unreasonable amount of effort to update, you win. They can't compel you to unreasonable effort as per their own TOS.

This assumes they even know that the model hasn't been updated. Who is this actually intended for? I'd bet it's for companies hosting the model. In those cases, the definition of reasonable effort is a little closer to "it'll break our stuff if we touch it" rather than "oh silly me, I forgot how to spell r-s-y-n-c".

Hosting companies can probably just claim they're covered under Section 230, and Google has to go bother the individual users, not them.

I don't believe that would apply if the host is curating the models they host.

Oh I tried to update, it's just that my router drops the connection after a few hundred MBs...

If you evaluate what it takes to update, and judge the effort unreasonable, that should be enough. Maybe make a powerpoint presenting that result, if you want something for the lawyers. If you don't see a way forward that leads to a result with reasonable effort you don't have to continue working on it until you hit some arbitrary threshold for unreasonable effort.

This sounds like a clause to cover themselves in case older versions have any serious issues

reasonable effort - meaning if their changes meaningfully impact my usage, negatively, it would be unreasonable to ask me to upgrade.

sounds good.

this is not financial advice and ianal.

Isn't this just lawyer speak for "we update our model a lot, and we've never signed off on saying we're going to support every previous release we've ever published, and may turn them off at any time, don't complain about it when we do."

We're talking about downloadable weights here, so they can't turn them off, or force you (through technical means) to use a newer version.

It's a local model, they can't turn it off. It's files on your computer without network access.

but what if they send a lawyer to ask firmly? (kindly, but firmly.)

They'd need to send a lot of lawyers, considering that they have no idea how many people are using the model, and very little way of finding out. And they'd need a TOS violation. It would be generally expensive for them to do at scale; this isn't about "turning it off" arbitrarily, it's a CYA in case someone specific does something really bad that makes Google look bad: Google can patch the model to make it not comply with the bad request, and then demand the person running the model update or else lose their license to use the product. It's a scalpel, not an off switch.

Ugh, I would fully expect this kind of clause to start popping up in other software ToSes soon if it hasn't already. Contractually mandatory automatic updates.

I appreciated this post clarifying the distinction between "open model" and "open source":

https://opensource.googleblog.com/2024/02/building-open-mode...

I'm not sure how to feel about the restrictions. "No porn" feels prudish, particularly for this millennium. I tend to err on the side of freedom in intellectual/political matters; however, the others seem fairly reasonable as far as restrictions go.

Huh. I wonder why is that a part of the terms. I feel like that's more of a support concern.

You don't have to agree to this policy to use the model.

model watermarking? does this exist?

Reply on news.ycombinator.com