Google tells staff it plans to limit publishing AI research | Hacker News Reader

Google tells staff it plans to limit publishing AI research

businessinsider.com

63 pointszone41128 comments

I thought this would happen: https://twitter.com/LechMazur/status/1617331853022711810, even though a Google Brain researcher said he didn't want it to happen: https://twitter.com/giffmana/status/1635724033307328517. It makes sense from a commercial perspective. This might slow down AI progress for those concerned about its risks.

Makes a lot of sense. Given that AI has gone commercial and might be the dominant technology of this/next decade, this makes a lot of sense.

I doubt there will be a big exodus. Where can researchers go who has both deep pockets for salary and infra and will let them publish the same way google was allowing.

OpenAI is anyways not publishing anything (see the GPT-4 pdf which is so void of any details).

I expect some amount of brain drain as researchers interested in publishing go elsewhere. That assumes other companies maintain an open publishing policy. I expect at least some will as it serves their interests (Meta for example).

I also think that these companies are realizing that at some stage, if AI keeps becoming more powerful, it may actually jeopardize their business, it's quite a conundrum for everyone.

Just as employees are feeling afraid for their jobs, I'm sure some of these companies, probably even Open AI themselves feel similar.

Like it's fun to release these models and keep the secret sauce secret, but at some stage, Open AI will probably be breached or someone will leak information about the architecture and weights, or both, and then it's kind of game over from there.

Odd to see this the day after the appearance of what's supposed to be a "leaked" memo from inside Google advocating the opposite course -- claiming that the outsider crowd playing with (mostly) LLaMa derivatives is getting good results, and that the best thing for Google might be to try to engage. It claims, for example, that the open-source crowd, trying to find cheaper training methods using smaller, more targeted data sets -- because they have to -- has found ones that work as well, by some metrics, as anything that the groups with huge compute clusters. And that because they work as well, with less resources, they can subsequently explore alternatives a whole lot quicker than anything involving training new base models from scratch.

The (supposed, unsourced) leak: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

This is different to publishing new research though ?

https://archive.is/seCfH

This is not surprising but at the same time, sad news for the community.

My thought is the following:

I think this is partly thanks to closedAI (openAI) but also partly Googles fault.

Google underestimated the Transformer model, closedAI did not and made something revolutionary with it, with little contribution back to the community (for example how little details they have given out regarding GPT-4).

Google realized this mistake and is now trying to catch up. They (Google) learned their lesson now.

My issue with this going forward is that Google might release some other powerful models / techniques as well and might underestimate them too, but this time, they will just collect dust somewhere in the dark.

Yea agreed. Thanks to OpenAI. Since OpenAI commercialized the technique but did not give back to the community, why should Google continue to publish their cutting edge research?

I sincerely hope this doesn't become a trend, it's dangerous and hurts the academic world a lot.

Sadly, that's the new reality now. Since the cat is out of the box, AI openness is no more. ClosedAI killed it the moment they decided to go chase the bag. The scary part is everything from here will be ruled by money before anything else.

Oddly enough, it may also have the reverse effect.

If the academic crowd stops relying on corporations and promote truly open/scalable tools, then this can start a virtuous cycle that promotes truly open solutions.

Having said that, I wouldn't hold my breath with the current state of academia though.

Frankly, they should be litigated for using the word Open in their official name.

Sounds like this is the end of DeepMind as we know it. The move will only push the last remnants of DeepMind out of Google, as there are no leverage or incentive to stay.

Salaries?

Is a thin defense in hot markets. There are always other players out there with deep pockets too

it is not clear if pockets are deep enough, there are lots of lay offs going through the industry.

Deep pockets AND freedom to publish?

This will be like many other industries:

Make 100k a year at this noble job and better the knowledge of mankind

Make 500k a year at a soulsucking corp job, doing research to better the rate at which we get people to click adds by 1%

Guess which jobs many great minds would take? I would take #2. At the end of the day, I can't feed my family with goodwill and back pats.

Is the assumption of 5x the pay going to hold though in light of:

https://news.ycombinator.com/item?id=35813322

Sounds to me like Google is butt hurt they got out played with their own research.

One benefit of publishing that this will diminish is cross examination and validation. Good luck living in your own bubble.

This is really sad. End of an golden era of AI research. Within few years all major companies will stop producing fundamental research. We are going to get lot of marketing material masquerading as research paper ( looking at you GPT4). Thanks ClosedAI ( OpenAI ) for brining so much openness in the field /s

This seems like it will result in a much more costly on-boarding process new employees.

Currently, if you hire some freshly minted phd to work on ML stuff, chances are they have read most the specialty's relevant cutting-edge research. Moving forward these same phds may have to spend months learning your fancy model architectures just to be somewhat useful. Hopefully at the end of that extended on-boarding, google doesn't find out their new hire is a complete dud. I guess if you have money, burn it.

Plus, if I spend years working on research at Google only to be unable to take those skills with me to my next job... well I wouldn't say yes to that job in the first place. Why not put a big sign on the door that proudly reads "lifers only"?

Sounds like the opposite of the response that https://www.semianalysis.com/p/google-we-have-no-moat-and-ne... seemed to advocate...

So the lesson they could only take from their failings is to limit publishing instead of capitalizing on new techs? Almost every if not all authors of "Attention is all you need" left Google before ChatGPT broke the world. If the paper was internal I suspect they would have probably left even earlier, taken their ideas elsewhere and still beaten Google to it. If everyone stayed hush hush, Google might not even know what gave other's an edge. There's no secret to AI as we know it today, deep learning, RL, transformers, LOTS and LOTS of data, plus lots and lots of training.

This is ending soon. Even according to sam altman from ai we have reached a limit on large model. We need new techniques like attention, transformer, layernorms

The thing that really surprised me is how this whole revolution happened and patents haven’t turned the field into a minefield yet.

Cooperation ends when competition starts

Reply on news.ycombinator.com