39 karma · joined August 15, 2024
Laws are difficult and nuanced. Deciding between yes/no takes me a long time, because I am not a lawyer or a legislator. Even for them, again, legislating is a job for a reason... I know I like certain things, and dislike others, I know basically which direction I want things to go; however, deciding whether a concrete bill is a reasonable-enough implementation that has closed loopholes is way out of the scope of decisions I would trust myself, or most citizens, to make well. This, it feels like a waste.
* LLM chess play is hyper sensitive to the harness being used (see: the section on regurgitation), and only mildly sensitive to fine tuning
* The best ELOs the author was observing were from 1500 to 1750 or so (circa 2024/25). Not grandmaster, but no longer incompetent monkey either.
But hey, they're actually good at chess if you prompt correctly so.... https://dynomight.net/more-chess/
the cross-organism increase in fidelity is pretty damn shocking. Ribosomes are one of the most optimized, slow-evolving things out there, and transplanting genes from animals to coli is already a very checkered endeavor.
This also relates to my interp of the article. There's grinding/training and then there's "having that dog in ya". He's trying to remind people to do the latter.
As for Kokotajlo, he's a good SF author, but I don't think it's nearly as good a prediction as it appears at first glance.
Like any SF, he's correct about some things, and way, way off base about others. Specifically, he was mostly correct about short-term (1-2 year horizon) technical aspects, but anyone familiar with how transformers work would have guessed those. I'm not even in AI/rationalia land, but believe I was having these exact conversations by 2021. Specifically about multimodality and steam-of-consciousness ['chain of thought' reasoning]. He is spot-on about chip shortages (to my eyes, his most well-done prediction).
Other thoughts are a storytelling device, like a well-told horoscope. He makes non-categorical statements that were already happening as if they categorically will become a new thing. But really he's just going off the current trend that everyone knows, and thus will be nearly-impossible to falsify. For example, 'People who still remember 2021 think of it as the golden days, when conformism and censorship and polarization were noticeably less than they are now. Just as it is normal for newspapers to have a bias/slant, it is normal for internet spaces of all kinds—forums, social networks, streams, podcasts, news aggregators, email clients—to have some degree of censorship (some set of ideas that are prohibited or at least down-weighted in the recommendation algorithms) and some degree of propaganda.' Sounds scary! But it is literally just saying 'polarization and marketing specialization will continue to deteriorate'.... which, duh. In essence, you could make this statement for 2018-> 2021, or for 2008->2018 and 'feel' just as correct!
Other key thoughts were somewhat off. Specifically, 'It now costs an order of magnitude less compute (compared to 2020) to pre-train a giant model' is kind-of-true, in that it costs less now, but the models have gotten so much bigger that the prices haven't actually gone down much. Might be changing soon, but definitely several years off.
The scariest central prediction that the political stuff rests on is 'After years of tinkering and incremental progress, AIs can now play Diplomacy as well as human experts'. This part, as far as I can tell, is just not true and hasn't changed.
And, well, 2026 predictions are just not correct at all. Some examples: 'Mormons create a “Christian Coalition” internet stack, complete with its own email client, social network, payment processor, news aggregator, etc.' -- basically untrue. I know a few ex-mormons, such a thing is not prolific.
'The US and many other Western governments are gears-locked' The US is far from gears-locked, in fact, it's full-steam ahead to er, well, something.
'“Should I wear shorts today?” “Order some more toothpaste, please. Oh, and also an air purifier.” “Is this cover letter professional-sounding?”' -- kind of correct, but in practice most people just do the last one. I'm not aware of many that give chatgpt agents credit card access (refer to the famous vending machine experiment). But predicting this with GPT3 is not that crazy, cause people were already doing that.
Personally, I am much more biased towards the METR timelines/way of thinking: https://www.lesswrong.com/posts/deesrjitvXM4xYGZd/metr-measu...
I found Kokotajilo's response comment to this kind of, idk, immature, and not especially mathematical.
Not really familiar with translation machinery or luciferase, but it seems pretty compelling to me.
There are some central concepts, and which ones matter to you depends on what you're planning to do with your future. The key points are:
* Matrices/vectors as grids of numbers/computational tools.
* Matrices/vectors as positions and transformations of those positions.
* Matrices/vectors as more abstract geometric objects
* Matrices/vectors as algebraic objects.
There is no 'done right' imo. My sense is LADW is probably the best option for a motivated honors math student, because unlike Axler the author doesn't hate determinants for whatever reason. Strang would work well for engineers. Axler is mostly concerned with the last two, but IMO this makes him kinda niche.
(A: there be dragons)