Ride the Hardware Lottery
pagestlabs.substack.com
pagestlabs.substack.com
"Hardware, systems and algorithms research communities have historically had different incentive structures and fluctuating motivation to engage with each other explicitly. This historical treatment is odd given that hardware and software have frequently determined which research ideas succeed (and fail). This essay introduces the term hardware lottery to describe when a research idea wins because it is suited to the available software and hardware and not because the idea is superior to alternative research directions. Examples from early computer science history illustrate how hardware lotteries can delay research progress by casting successful ideas as failures. These lessons are particularly salient given the advent of domain specialized hardware which make it increasingly costly to stray off of the beaten path of research ideas. This essay posits that the gains from progress in computing are likely to become even more uneven, with certain research directions moving into the fast-lane while progress on others is further obstructed."
Are there any examples where a VLP model actually transcends the limits of the human brain?
Other than that… well, I don’t even know of any other VLP models. But there are other examples of ML models in general transcending humans, like Watson in Jeopardy, and AlphaGo.
In the context of AI, Bert already had knowledge embedded into it [1], and it's not considered a VLP model afaik. So it's more of a quantitative improvement, contrary to the claim of the author that there's a qualitative difference between VLP models and everything else.
[1] https://syncedreview.com/2020/08/12/how-smart-is-bert-evalua...