> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access.
Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.
Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.
You seem to have changed your stance 180° in the middle of your comment.
This is the first time I've heard this. This kind of thing definitely needs more news coverage.
What I would like to see next is: to extend the corpus with task specific data where available, making it more performant on these specific tasks; to have a searchable memory module (ability to retrieve text from its corpus or new text additions); to enlarge its context window; to enlarge the corpus, including more non-english content; to be trained on more than just text: images, audio and video, for grounding. It also needs a leash - a second model to stop it from generating profanity or insulting things and better ability to control the expected size and format of the generated text: dialogue, article, novel, etc.
The goal should be pushing science forward, not maximizing brand value. Enabling top scientists in the field to reproduce and probe your work is not "pandering". It's participating in the scientific process.
Clearly, the point I'm making in the top comment is the following: OpenAI apparently restricts access to people -- even very highly respective scientists -- who happen to be critical of their previous work. It's impossible to prove intent, but limiting access for people like Gary Marcus is while giving access to tech journalists likely to write breathless hype pieces is odd behavior that deserves to be criticized.
This is how winter happens and is a decidedly anti-scientific mindset.
Do you consult with non-technical/low-technical execs on AI investments? If you did, the answer to your question would be obvious. For every dollar Microsoft and VC firms can spend on "we're the best technical minds in the world" Research&Marketing departments, there are thousands dollars in the rest of the economy that can only be captured by ensuring execs make prudent AI investments. Step zero is always deconstructing the hype around GPT/Alpha*/etc.
Well, who is Gary Marcus anyway? I looked up his publications in the last 2 years on Google Scholar and I see no hard contributions, all he has are some publications about futurology and AI critique. His wiki page says he's a cognitive scientist who once sold an ML company to Uber, but not an AI expert.
Why didn't Gary invent a better language model to show us how it's done? If he knows better than the guys at OpenAI, let him show the path ahead. When someone has superior results it's not constructive to throw bullshit at their accomplishments.