What does the future of open-source large neural nets look like? My understanding is GPT-3 takes ~600GB of GPU memory to run inference. Does an open source model just allow you a choice of a handful of cloud providers instead of one?
Open source doesn’t mean that everyone will be rolling their own. It means that lots of players will start to offer endpoints with GPT-X, perhaps bundled with other services. It is good for the market.
Right, this is why a cloud company is sponsoring the replication. They can’t sell GPT-3 as a service until they get a copy of it.