Hacker News Reader

Comment by akyu | Hacker News Reader

My conspiracy theory on this: This entire fiasco is actually an experiment to gauge public reaction to this kind of release strategy. Eventually the point will come where the choice of releasing a model may have real ethical considerations, so they are test driving the possibilities with a relatively harmless model.

Given the, in my opinion, huge overreaction to all of this, I fear this may only encourage AI research groups to be more secretive with their work.

View on HN

That's pretty much what they are publicly saying, right?

> This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas.

Ah I missed that.

they acknowledged even in the original blog post that any well-funded group of NLP researchers would be able to replicate their work within a few weeks/months (including whatever corporation or state or terrorist group you worry about), and that in terms of methodology it is a natural, incremental improvement over existing techniques. so it's sort of obvious that withholding release can't prevent any harm.

the good faith reason to withhold release is as you say -- start a conversation now about research norms so researchers have some decision making framework in case they come up with something really surprisingly dangerous.

the bad faith reason is it gives them great PR. this was surely part of the motivation, but I bet OpenAI didn't quite expect the level of derangement in the articles that got published, and may regret it a little bit.

Can we even say for certain it's an improvement over say something like TransformerXL? As far as I could see, the changes over GPT were a couple extra and tweak to layer normalizations, a small change to initialization and a change to text pre-processing. Other than for pre-processing, I didn't catch anything on theoretical motivations for these choices nor anything on ablation studies. The only thing that can be said for certain is it used lots of data and a very large number of parameters, trained on powerful hardware and achieved unmatched results in natural language generation.

From their blog post:

>These samples have substantial policy implications: large language models are becoming increasingly easy to steer towards scalable, customized, coherent text generation, which in turn could be used in a number of beneficial as well as malicious ways. We’ll discuss these implications below in more detail, and outline a publication experiment we are taking in light of such considerations.

>a publication experiment we are taking in light of such considerations.

So I think you're correct, they aren't really too scared about this one, but they realize they might be soon. Seeing how this turns out (how quickly its replicated, how quickly someone makes a wrapper for it that you can download and run yourself in minutes) will inform for more serious situations in the future.

When you have whole teams doing "AI safety" at FANG, what do you think would be a logical output of their work? Wouldn't it correlate with what we see from OpenAI?

OpenAI is just trying to make a buck. Just like most other programmers, they don't give a damn about how damaging their code is going to be.

This is just false. Cynicism isn't a substitute for knowledge.

OpenAI is literally a non-profit.

Not saying I agree with the parent but while OpenAI is non-profit it's owned by billionaires whose companies directly benefit from their research.

Perhaps they kept it closed source because they sold it to one of those companies? Non-profits are allowed to sell things, after all.

Do you think that means they don't get paid? Non-profit itself isn't a super meaningful term as far as if profits are being made or not. To be a non-profit, many hoops have to be jumped through, certain forms of profit have to be zero (I don't know the specifics, but a friend of mine was considering starting a non-profit at one point) but the employees are still paid.

You can literally make millions of dollars working for non-profits. The highest paid CEOs make several million dollars per year and the biggest non-profits have annual budgets in the billions. The annual revenue from all non-profits in the US is in the trillions.

The NFL is a nonprofit organization too...

Reply on news.ycombinator.com