Software expands to fill the available resources. If you want more efficient software, build it on less powerful hardware. AI training runs are no exception!
Software today require several dozen gigabytes of RAM and two dozen CPU cores just to render some simple fucking text because everyone has several dozen gigabytes of RAM and two dozen CPU cores.
It’s not altogether different from how everything seems to be much more expensive than it was in the 1950s. There are many real and concerning reasons for that, but one less concerning reason is that we have higher standards and expectations now, and that translates to higher cost.
To give a random example, Apache http server for windows is less than 12mb download. I used to download Oracle's http which I'm told is based on that, but that's like 1GB++ haven't checked in recent years.
Because their Engineers are constantly in situations were they have to work with much less resources. And they come up with stuff that constantly surprises people for being much cheaper. The western reaction is to then go and buy them out. Its not going to work forever. They have already taken over large swathes of the tech landscape.
Even with AI for example, Real Time inference is not required in majority of cases.
But the western engineer and corp exec in big tech, are used to buying new data centers everyday (not because of specific Customer Demand, but because of intent to capture market/snuff out competition/build moats or what ever bs an environment of over abundance has trained them to do) and then they have zillions of machines idling which they use to provide real-time inference raising the cost for everyone.
Sooner or later someone in China or India working on non real time inference will offer large corps much cheaper solutions.
The Western model is where you forget how to cook a meal at home, and end up relying on McDonalds for food, because they are everywhere, thanks to their prime directive of survival via opening new stores everyday.
Make those fucking assholes use the hardware of the people instead of some monstrosity with a 256 cores CPU and 48 TB of RAM and 24 exabytes of enterprise SSD and an RTX Cinco Grande connected via optical fiber to the Amazonflare Cloudnet. We will see lean and mean software literally overnight.
Are you sure it’s the engineers who are being complacent? I know quite a few coders who’d love nothing more than to spend months on optimizations eking out small performance gains. Their bosses (or the market) don’t let them.
If shipping fast means you spend an extra $1000 once per developer, guess what almost every company is going to do?
The real fucking assholes are the ones that enable the fucking assholes you refer to.
So there is a use for fat development stations. Increases my productivity which is very important as I am an independent vendor and pay for all my tools.
That should knock God knows how many GPU cycles, time and training of models.
It’s much harder if anything, due to the sheer amount of topics censored.
But I suspect it applies to citizen facing things. It doesn't likely hold up research or pressures them into returning POC Viking images.
I could be wrong of course.
If you have to do political policing one way or the other, you'll have to invest resources to achieve it. On the other hand, maybe it's beneficial for r&d because it pressures you to do hard things that your product benefits from, e.g. controllability in ML-products.
Even with latest hardware, there must be incentives to optimise on size, memory, CPU time etc. But given natural tendency, we just optimise on what's the rarest of them all - time to market. Get it shipping fast.
I was once in a training (as new consultant) for software, the instructor, an old fashioned guy said quit using the mouse learn the keyboard shortcuts - your customer is watching you. He was damn right cos years later, I realised so many customers did remark, how do you do that so fast? To this day, running some command in Excel (say) access Name manager, I use kB shortcut that hasn't changed in decades. I frankly even lost track of where to find them in the menu bar.
Simple things but powerful. Do with less.
1. While building https://gitpodcast.com
2. Code snip: https://github.com/BandarLabs/gitpodcast/blob/main/backend/a...
To make an analogy, that is why I think even with AI work expands to fill available resources (human+AI). I don't think jobless rate will be high, instead we will see demand expansion.
They write software that usurps other people's computing resources, e.g., CPU, storage and an internet connection that they do not pay for.
It's one reason I use NetBSD, custom barebones Linux. I write and compile software on single core old computers and cheap eMMC laptops.
There is a lot of fun to be had working with vintage hardware in spare time though