The future of foundation models is closed-source
blog.johnluttig.com
blog.johnluttig.com
So Linux is a simple act of charity from one person. If the author so fundamentally misunderstands what Open Source software is in principle and Linux is in particular then I struggle to see how he's qualified to come to such a definitive conclusion on a subject as complex as foundation models.
I don't find their opinions worth considering all too deeply, given that bias.
If closed-source consolidation of models happens, it'll be because capitalism encourages and incentivizes centralization and secrecy, not because of any other reasons the author outlines.
Which is extremely bizarre, because the most legitimate criticisms of socialism are its tendency to revert to authoritarianism or otherwise impose crooked rules on unconsenting people.
Whereas open source is... completely voluntary? It's like being opposed to private community organizations because a co-op might break the stranglehold a monopolist has on a market. The only one who doesn't want that is the monopolist.
Which is, incidentally, the reason the US now has a bunch of crooked laws inhibiting non-profits from engaging in things that compete with businesses, e.g. by providing a service for a competitive price near cost rather than entirely for free solely out of donations.
"I admit I have an allergic reaction when many open-source advocates expose their socialist tendencies from Europe, academia, or both. ... America's tech success is subject to endless criticism from those who missed out, but we handily won the last tech wave because American capitalism aligns users and companies for the long term."
I'd take everything else in this article with a huge grain of salt.
So yeah LeCunn doesn’t want a small number of private companies to control our information. I wonder if the author views that as “socialist”.
I've lived on the west coast of the US for 20 years now, and the prospect of this terrifies me. In a "humanity's future is bleak" sort of way.
> an unusual open-source alliance has formed among developers who want handouts
Handouts? Really? I've been an open source user and developer for decades now, and... "handouts"? I'm not even sure what to say to that.
At this point in my life I'm convinced that the style of capitalism practiced in the US is a failure. It gives you a lot of initial gains (which are, no doubt about it, quite important) but ultimately creates an untenable, inhumane situation for lots of people. I'm not convinced that European-style socialism (whatever that is; it's not like all the European countries are united on that front) is the answer either, though. I think a unique blend of capitalism and socialism is likely the only way forward without further wealth concentration and income inequality (not to mention planetary-scale ecological destruction, since the current forms of capitalism don't incentivize us to pursue sustainability in the long term). I'm not sure what that blend is, and I'm increasingly pessimistic that the people in power will allow us to get there, even if the alternatives will be disastrous for them as well.
In some sense this is right, at least as long as all the research we're replicating boils down to "hey if you double the GPUs and data you get double the model quality". But whenever you make the "shit costs money" argument, you also entertain a moralistic co-argument: "I am entitled to supracompetitive profit". And this guy is definitely on the side of "pay up, capitalism works, you pinko commies".
The problem with this argument is that the datasets we currently train on are already stolen data. With the exception of extant FOSS licensing, which probably allow training[1], of course. The argument for closed models boils down to "we are entitled to be paid for our model but the artists and writers we stole[2] from are not".
[0] The clarion call of any savvy fascist
[1] Good luck figuring out how to comply with CC-BY-SA though
[2] In the Valentian sense where recording a movie onto a VHS tape makes you a home-invading rapist.
The race is on behind the scenes, and the dollar valuations are proof of it. This essay starts a worthwhile conversation about the nature of the reproducible builds beast, but has little to bring in the way of insights or revelations IMHO
Whether people will eventually give all their data to one of four companies is another matter. For example, it depends a lot on how much further improvements go before we reach diminishing returns and end the current hype cycle. Chips might catch up with how much compute vs. how much (good quality) data you need to build a useful model. I also think it's not entirely impossible that state actors might not go for being ruled by one of four US companies.