170 karma · joined May 3, 2022
https://www.theguardian.com/technology/2025/may/16/elon-musk...
Similarly with LLMs, it's good tech, but I do not believe for a single second that these things will ever replace anyone who is remotely skilled. People will be augmented by them. They won't be fully replaced. Well, I hope not anyway.
I predict costs will fall.
Cost is a big factor - i really want to make models that can run on average CPU only machines so most of the world can benefit, rather than needing expensive GPUs or an internet connection + subscriptions. Another big factor is privacy (you don't need to trust a 3rd party with your inputs).
As for the hyperparameters, pure bruteforce trial and error. It feels more like a dark art than a science. You roll the dice and then start tweaking things until the loss looks like it's dropping nicely and consistently, and the checkpoints are starting to output things resembling what we want. I sometimes do inference using checkpoints just to get a feel of if the model is learning (regardless of loss)
https://www.nytimes.com/2025/07/15/opinion/israel-gaza-holoc...
And let’s not pretend history started on October 7th, that region has been in turmoil for almost a century now, so trying to excuse the sniping of children in the head, mass starvation, subjugation of a whole population, straight up ethnic cleansing in this day and age is indefensible no matter what happened. You say you’re not excusing but only explaining but I don’t want to hear any explanation that equates babies as threats that are okay to kill in war. Hamas could’ve dropped a nuke on Tel Aviv and it still wouldn’t excuse sniping innocent children in the head. Did it become okay to liquidate the ghettoised Jews because they struggled for their freedom? They tried to rise up too? They weren’t prepared to lay down without a fight. And where is the outrage against the Israeli government for firing on their own citizens at the festival?