Hacker News Reader

Comment by juanbyrge | Hacker News Reader

I was a little disappointed in the quality of this research. I can see where Google is coming from. Some of these topics seem regurgitated (e.g. bias in online text) and some are just irrelevant (CO2 emissions). Training language models does not contribute in any meaningful way to global CO2 emissions. Overall not a very strong paper in contrast to the other reactions I've been seeing online.

View on HN

> I was little disappointed in the quality of this research

But the research is not even published yet

Kinda the point though isn't it?

We're only reading a summary of the paper because apparently the authors aren't confident enough in its quality to release it publicly.

You can't claim that Google dismissed this paper out-of-hand while simultaneously saying "oh, but it's too much of a draft to release publicly". Uh, if it was too much of a draft for the public why shouldn't it be too drafty for Google? Are we really pretending that Google has lower standards of quality than the general public?

That, or Google is censoring the paper because of the conclusion it comes to, which doesn't align with their business interests.

If a company is paying a researcher a lofty salary (I am guessing ballpark mid six figures), and provides that researcher virtually unlimited tools and resources, shouldn't that company have a say in what material gets published by the researcher? Why shouldn't it have the ability to restrict research of dubious quality that makes borderline incorrect conclusions about the company?

I’m taking the article at face value, which states the authors think it’s too much of a draft.

I naively expected this article to include a link to the paper somewhere in it, so we could decide for ourselves. Oh well.

Training language models does not contribute in any meaningful way to global CO2 emissions.

Google consumes a vast amount of energy: "10.6 terawatt hours in 2018, up from 2.86 terawatt hours in 2011."[1]

If training and retraining models is a significant and inefficient part of google's energy consumption, the point doesn't seem insignificant(edit: The most advanced AI model involve as much as computing and energy any programs ever created [2], btw). I'm biased by the impression Google's actual search results haven't improved very much but I don't think I'm alone in that impression.

[1] https://www.statista.com/statistics/788540/energy-consumptio...

[2] https://openai.com/blog/ai-and-compute/

I obviously don't have the data, but I would argue that a tiny tiny fraction of Google's energy usage is dedicated to training language models.

I'm guessing the vast majority of the energy usage is for serving billions of requests for various products, such as search, youtube, maps, gmail, photos, etc.., and the cpu, network, and storage requirements for those requests.

Training and retraining ML models is definitely not on the hot path.

Reply on news.ycombinator.com