joshjob42

If you don't want them to have a history only communicate via disappearing messages.

I'm not vegetarian or vegan, but I think eating meat is morally terrible, I just can't be bothered to go through all the trouble to not eat or use animal products when they're everywhere, cheap, and taste fantastic. But I enjoy Impossible's products immensely, and follow the artificial meat and dairy field closely, and have been long a champion of these efforts so as to make it easy to not cause any harm to animals rather than it being a royal pain in the ass.

Mapping is expensive, but not really in a per-mile-driven basis. There are 4M miles of public roads which get over 3.2T miles of total driving, or ~800k vehicle-miles per road-mile. You could have pretty high mapping costs per mile and still have very low per-mile-traveled costs for mapping. And there's every reason to think the cost of mapping and updating maps on a per-road-mile basis will go down over time, not up.

Waymo is scaling pretty rapidly, and the rate of expansion is accelerating. They've been proving out the technology, and are only now starting to commission special purpose vehicles as they move out of the research phase and into deployment.

Perhaps Tesla will catch up, but for now we know Waymo's are at least ~6x safer than humans in diverse independent conditions, while FSD according to publicly available information is 50-100x less safe (with critical interventions every few hundred miles).

Another way for intelligence to get too cheap to meter is for the cost to fall so low it becomes hyperabundant. If you were to, for instance, take AI2027 as a benchmark and think ultimately we'll achieve something like the equivalent of John von Neumann in a box with a 2T dense equivalent parameter model and it will match such a Nobel prize winner's productivity when running inference at say 15 tokens a second (as fast as people can read) then you only need in principle 60 teraflops of AI infernce compute, which is roughly 2x the current Apple Neural Engine. So plausibly by the time you get to the 2030s, every laptop, smartphone, etc will be easily able to run models as powerful as the smartest people.

Somewhat longer term, I'm sure Altman expects the entire process to be automated and for the computational efficiency to rise significantly. If you take recent estimates from various players in the reversible computing space, you'd guesstimate that you ought to be able to do 60tflops by the late 2030s using under 0.1W or ~1kWh/yr which Helion could produce for ~1¢. I do feel like 1 year of cognitive labor from the smartest person a penny or two renders intelligence too cheap to meter out on a per-hour basis.

A frequentist arguably would say the question doesn't really have any meaning since probabilities are about long run frequencies of things occuring. They might do various tests or tell you the probability of that outcome under various probabilities for heads.

A Bayesian would make an initial assumption about the probability of any given probability, and then compute a posterior using the likelihood function the frequentist may have, and give you a distribution for what you should believe about the what the true probability of heads is on your next coin toss.

In general, the latter is more meaningful and informative. There's also pretty good arguments that any coherent method of representing credences is isomorphic to probability, see Cox's theorem.

This is a big improvement over numpy, but I don't see much of a compelling reason to go back to Python.

There already is a LispSyntax.jl but it's sort of trying to be a Clojure clone and I don't care for it, haha.

It is pretty REPL focused, but what is the thing you want that you don't have? Like better debugger support in VSCode or something?

You use a laptop, you probably play some video games, you see movies, you probably eat meat, you probably drive a car around town instead of a velomobile.

So the question is, how much energy is used by AI and what does it get us? Google's latest TPUs consume ~250W per fp8-petaflop at the data-center level (ie including cooling etc, according to their recent presentations). Even assuming 50% utilization which is a bit poor, that's say 0.5kW per fp8-petaflop, or ~1kW per 10^15 dense-parameter-equivalent tokens per second. So using a huge model, say 10T dense-equivalent parameters, and doing inference at the rate of o3-mini or Gemini 2.5 Pro (around 150 tokens a second) consumes 1.5kW during generation time. But maybe a better way to think about it is just that you'd be using 10J of energy per token, or ~15J of energy per word.

In that context then, generating a book the length of Fellowship of the Ring (~180k words) would consume ~0.75kWh of energy. That's about like playing a video game on a PS5 for ~3 hours. In the US, that's on average ~270g of CO2, like driving ~1.5 miles or so in a Prius. It'd be about like riding ~30 miles on an e-bike.

Another way to think of it might be that if you are texting with a friend, wherein lets say you both aggressively type at ~120wpm, you'd be using ~15W of power to chat with an AI at a similar pace, about the power draw of your friend's Macbook Air he would be typing on. That's around 0.5-0.7 miles on an ebike per hour of chatting.

So even compared to your typical leisure activities, chatting with an AI is comparable in resource use to whatever else you'd be doing in that time most likely. And of course, a human in the US on average generates ~1.5kg of CO2 per hour they're alive in the US. In Bolivia, it's ~150g/hr, and in Sudan it's ~50g/hr or so. So if you're a company and you want to be climate conscious merely hiring someone who is a ~ subsistence farmer to do a job such that their standard of living rises enough to roughly emit the CO2 of the average Bolivian would mean they emit more carbon in a typical day than running an AI to produce as many words as are in the entire Game of Thrones book series to date. And if you were going to help someone immigrate from Bolivia to the US, you could have the AI write ~10 Songs of Ice and Fire a day for the same net CO2 output.

Not to say that we shouldn't do those things because of climate or whatever, but I'm just saying that the objective energy and climate impact of using these models for things is small compared to basically any alternative for completing a task and most entertainment activities people do (movie theaters for instance use ~7kW or so while running, so in the time you spend watching the Fellowship of the Ring in a theater, an AI could write a text as long as FOTR 28 times).

Let's assume you have everything in the Culture except the ASIs. So you have full blown Drexlerian nanotechnology, access to all the technology more or less possible out to the limits of physical law. Energy is functionally infinite, and if you want basically anything to be produced you simply order it from the AI equivalent of ChatGPT today and it is directly manufactured from base chemical feedstocks and delivered to you by a simple automaton. Food, clothes, furniture, any sort of electronics. Homes are extremely plentiful and similarly require ~ no effort to obtain, since they are just snapped together from atomically perfect panels. When you're done with something, it gets recycled back to feedstocks to be made into something else.

Okay. Now what? You're free to love who you like, befriend who you like. But nothing you do matters really for anyone else beyond sentimental value, since nothing you do effects whether they live or die (you're all functionally immortal after all, plausibly you can backup your consciousness and get resleeved if something truly catastrophic was to happen to you). There's no scarcity, no need to horde things, you play games and amuse yourself in whatever little hobbies delight you, and you socialize with those who you care to.

What exactly is the grand thing that you think people are going to be doing in that scenario that they can't do in the Culture?

It seems apparent that for the overwhelmingly enormous majority of people the Culture is identical to the case where there are no Minds, because to whatever extent some tiny portion of humans needs to make some sort of major decisions in a post-scarcity world with no Minds, 99.9999+% of people will not be among them. We see mostly people interacting with Special Circumstances (a tiny fraction of the overall population) and most of those who live on GSVs etc managed by Minds in SC themselves are also just going about their days and not involved with whatever machinations the Minds have going on.

Like, what is the problem that ASI really introduces here for regular people? And in a world where you are indeed free to leave the Culture (as we've seen) if you really want to, and there's plenty of variety in how smaller groups in the Culture live their lives, what exactly is missing? There is no oppression, no brutality, no war, disease, famine, poverty, or want and you're free to do as you like except kill people without their consent and/or otherwise hurt others. What more freedom do you really want?

But if you sold the Spark EV for 20k today with like 120mi of range, it would be perfect and would satisfy all my needs 99% of the time. Even mine (13k all in) was great here in LA with ~60mi of range. I loved how small and easy to park it was without feeling cramped to me at all. If it had CarPlay I'd've said it was the perfect car haha.

It's a shame they haven't rebooted it yet as a pure EV. It's right there in the name!

I'm skeptical of the claim that local is literally faster, but it's not impossible in the way you suggest.

In section 4 they discuss their projections specifically for model size, the state of inference chips in 2027, etc. It's largely pretty in line with expectations in terms of the capacity, and they only project them using 10k of their latest gen wafer scale inference chips by late 2027, roughly like 1M H100 equivalents. That doesn't seem at all impossible. They also earlier on discuss expectations for growth in efficiency of chips, and for growth in spending, which is only ~10x over the next 2.5 years, not unreasonable in absolute terms at all given the many tens of billions of dollars flooding in.

So on the "can we train the AI" front, they mostly are just projecting 2.5 years of the growth in scale we've been seeing.

The reason they predict a fairly hard takeoff is they expect that distillation, some algorithmic improvements, and iterated creation of synthetic data, training, and then making more synthetic data will enable significant improvements in efficiency of the underlying models (something still largely in line with developments over the last 2 years). In particular they expect a 10T parameter model in early 2027 to be basically human equivalent, and they expect it to "think" at about the rate humans do, 10 words/second. That would require ~300 teraflops of compute per second to think at that rate, or ~0.1H100e. That means one of their inference chips could potentially run ~1000 copies (or fewer copies faster etc. etc.) and thus they have the capacity for millions of human equivalent researchers (or 100k 40x speed researchers) in early 2027.

They further expect distillation of such models etc. to squeeze the necessary size down / more expensive models overseeing much smaller but still good models squeezing the effective amount of compute necessary, down to just 2T parameters and ~60 teraflops each, or 5000 human-equivalents per inference chip, making for up to 50M human-equivalents by late 2027.

This is probably the biggest open question and the place where the most criticism seems to me to be warranted. Their hardware timelines are pretty reasonable, but one could easily expect needing 10-100x more compute or even perhaps 1000x than they describe to achieve Nobel-winner AGI or superintelligence.

If basically a transformer, that means it needs at inference time ~200T flops per token. The paper assumes humans "think" at ~15 tokens/second which is about 10 words, similar to the reading speed of a college graduate. So that would be ~3 petaflops of compute per second.

Assuming that's fp8, an H100 could do ~4 petaflops, and the authors of AI 2027 guesstimate that purpose wafer scale inference chips circa late 2027 should be able to do ~400petaflops for inference, ~100 H100s worth, for ~$600k each for fabrication and installation into a datacenter.

Rounding that basically means ~$6k would buy you the compute to "think" at 10 words/second. Generally speaking that'd probably work out to maybe $3k/yr after depreciation and electricity costs, or ~30-50¢/hr of "human thought equivalent" 10 words/second. Running an AI at 50x human speed 24/7 would cost ~$23k/yr, so 1 OpenBrain researcher's salary could give them a team of ~10-20 such AIs running flat out all the time. Even if you think the AI would need an "extra" 10 or even 100x in terms of tokens/second to match humans, that still puts you at genius level AIs in principle runnable at human speed for 0.1 to 1x the median US income.

There's an open question whether training such a model is feasible in a few years, but the raw compute capability at the chip level to plausibly run a model that large at enormous speed at low cost is already existent (at the street price of B200's it'd cost ~$2-4/hr-human-equivalent).