int_19h

Selection works on the level of the genes. Dying without procreating can still be advantageous if doing so means that enough of your relatives survive, for example. https://en.wikipedia.org/wiki/Kin_selection

There's a converse effect too, though - because conservatives tend to attack modernist and postmodernist art, liberals tend to defend even things that are too ridiculous to be seriously defended.

It's not "fascism" to wince and shrug at the taped banana. It's just common sense. But if you insist that everyone who does so is secretly motivated by some kind of bigotry, you end up pushing them towards the right wingers anyway.

Ugly is in the right of the beholder when it comes to individuals, but societies also have collective preferences.

Stalin was notoriously conservative in most things. I mean, the guy banned abortions, criminalized homosexuality, and re-instituted paid high school.

It does let you do that, but only on platforms where they can enforce HDCP. So e.g. Edge on Windows or Safari on macOS work fine, and I think Chrome recently joined the list too. But not on Linux.

Herbert did genuinely believe that people overly relying on machines is problematic insofar as it "trains the mind" to think like a machine, and in the process loses the indeterminacy that he believed to be an inherent and valuable trait of human thinking that machines cannot replicate.

(Coincidentally, this is also the main threat in the original books, just manifesting in a different form, with prescient individuals "locking in" the rest of humanity.)

> These two things are surely related eh?

Not really, no? If anything AI is the tech that is the most likely to disrupt this economic system beyond the point of recovery.

> What's the difference between these?

The difference is that one is a piece of hardware, and another is the person deciding what that hardware will do, and to what ends.

> unless you, uh, nationalize the data centers at which point...

The data centers are not the problem. The fact that you need to be employed to have any kind of decent living at the same time when automation is getting good enough to displace people en masse across most industries is the problem.

You don't even need to go full socialist to solve that! Just tax AI providers on their revenue proportional to the number of jobs displaced by it, and use those funds as a pension for the displaced workers.

Killing things is literally the sole purpose of guns, though.

And even then we still need them because sometimes you do need to kill someone who's trying to kill you, which is not morally wrong.

The question, again, is how they are used.

The investors don't manufacture those higher quality goods and services - their workers do. The investors collect economic rent from the worker's outputs.

> It’s a bit of a self-serving argument, don’t you think?

I would call it more of a self-selecting one. Anthropic is basically hiring people with that mentality. I'm pretty sure that most of them do sincerely believe it, too. I'm skeptical about Dario himself though. The man had an opportunity to show moral backbone, and failed to do so; why should I trust him on that again?

> And how does that relate to the ask for oligopoly licensing within global democracy?

They are basically saying that they'll stop if everybody else does, which requires some kind of global enforcement mechanism.

> “We must build the nuclear bomb first in order to make sure no one else builds one.” This the most nonsense, disingenuous argument imaginable.

The difference between nuclear bomb and AGI (as understood by the likes of Anthropic) is that the latter triggers the technological singularity that renders any runner-ups moot. That is, so long as AGI is developed, we're going to get our robot overlords either way, but whoever gets there first gets to define their ethical system. If that is one's perspective, and if one sincerely believes that they are the only ones who can do it right, it's a coherent argument. It's just that the premises are very arrogant.

Your analogy with nukes actually works better for the position that AI development needs to be unconstrained because otherwise we'll lose the arms race to China. That is basically a repeat of https://en.wikipedia.org/wiki/Einstein%E2%80%93Szilard_lette.... I honestly don't know where I am on this. Realistically, if AI is indeed a power multiplier - and with all the recent security stuff it's hard to not see it that way - then an arms race feels inevitable, especially given the current worldwide political situation. I could believe in sincere international cooperation on this back in 1990s, but there's way too much saber rattling all around for it to work (and note that this goes both ways, i.e. China can similarly not be certain that US isn't secretly developing more powerful AI even if we do publicly announce a freeze).

That is not a counterargument to cloud inference though. You can also run open weight models in the cloud, and it's still cheaper. So privacy really is the only motivation to run on local hardware.

You can have abstract trust in democracy over autocracy, but this is not an abstract question - these are the actual choices. Does it really matter if the person who will use AI to fuck us has a "democratic mandate" to do so? We're fucked either way.

I have something like that running locally for my agentic harness (which includes cross-model messaging). There's a dedicated "product manager" session for it, and all other PMs are instructed to report issues with the harness as they occur to that session, while it is tasked to automatically prioritize and address them and coordinate fix deployment with other running sessions.

It works surprisingly well. The errors fixed are both genuine errors in the harness itself, but increasingly so upstream bugs (in the underlying agent apps like Codex, or in Herdr, which is used to expose uniform programmatic access to all those different apps) for which it needs to come up with workarounds. No regressions so far.

The cost is hard to judge on a subscription, especially when you're running really heavy tasks otherwise that dwarf any harness work.

I strongly disagree. I'm working on a semantic model for Lojban, heavily AI assisted, using multiple models. I have basically all popular frontier models doing research and panel debates. Astra and Fable are both noticeably ahead of everything else including their previous iterations. When it comes to reviews, they can also find more issues in others (or even their own) code.

I don't think you can ever make a case for it on cost savings in general. Inference is very obviously the kind of problem where things are cheaper at scale, and this is still true for smaller models.

The only reason to run locally is privacy.

The difference is still vast, it's just that the smaller stuff is "good enough" for many things now.

This is very misleading. There is no Chinese model currently that is on par with Fable or Astra. Kimi K3 is pretty good, but it's not that good.

Furthermore the Chinese models in that class are appropriately large. K3, for example, is a 2.8T-parameter model. Qwen 3.8 Max, another comparable model, is 2.4T params. Even with MoE, these are not "designed to run on pretty unimpressive hardware". Stuff like Qwen3.7-27B is, but it is also not even in the same ballpark as Opus, never mind frontier.

On chemical weapons, most large state actors have got rid of them. Not because of any moral reasons, though, but simply because they don't actually work all that well in modern conventional warfare. This is then framed as an ethical issue, but you only need to look at where the same countries stand on e.g. landmines to realize how much bullshit it all is.

Conversely, where you do still see chemical weapons used, it's usually asymmetric conflicts where "just gas the rebels" actually works much of the time and is much cheaper than other options. Big guys can afford the other options though, someone like Assad, not so much.

More on this: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...

Honestly? I'd rather al Qaeda have access to something like Fable, than the US government. The US government has capacity to do immensely more damage.

Anthropic specifically is basically saying that they believe it's even more dangerous if someone else gets to AGI before they do, so they have to either stop everyone or not stop themselves.

I personally disagree with that take - and, as you note, it's hard to take seriously ethical wrangles from a company that literally sued the government in court to allow their models to be used by Palantir of all people. But if one genuinely believes that it's the robots themselves (rather than the people controlling the robots) that will kill us all, it's not inconsistent.

> the rent-seeking parties

How do you think billionaires become billionaires?

Given the scale of the buildout, I think it would be sensible to mandate that, for every data center built, the companies doing that must also subsidize a new green power plant somewhere (not necessarily the same region, but where it makes the most sense), proportionally to consumption. Basically each kWh of power has to be compensated. No gas or any other fossil fuels even if there's pre-existing supply.

Separately there's a problem with the lack of enforcement, most notoriously Musk's infamous gas-powered data center that apparently violates a bunch of regulations including noise. Musk does this because he believes that he can get away with it in a sense that the eventual fines, if any, will be dwarfed by the profit extracted. The penalties need to be hefty enough that this computation doesn't work out.

> People show up and make decisions. They should then be able to leave and assume those decisions will be put into action.

But this isn't the case. In every representative democracy around the world, including ours, the bedrock principle is that elected politicians "vote their conscience", i.e. they are not bound by what they said they'll do if elected. So in practice you can't assume that what you voted for will be put into action. If anything, the track record on that is pretty poor.

OP specifically said "we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI" though. And I don't know how you get robot farms without that.

Of course it is, because our society is pretty much structured to place sociopaths on top. Why is it at all surprising that any technological advancement is going to be used by them in a way that doesn't benefit the rest of us?

But it's equally obvious that this has nothing to do with the tech itself.

You can still pursue understanding of things, full automation doesn't preclude that.

What it does is make it a choice rather than a survival necessity. The latter is where the "suffering" comes from.

We still are automating the process, it's just that the automated part of the process covers much of what we regularly do.

> Sure, the next step is complete regression to feudalism, but they aren't thinking that far ahead,

They are though! They are literally building castles for themselves already.

ju'i .atencon.

ju'i ro tcurista .e ro naltekno lukpipyprenu

.i le kompymacni na se pilno fi lo nu selfingypoksi kei .e lo nu mijgrabi

.i va'onai frili fa lo nu cnapygau le spinyverka kei .e lo nu blovyfuzli kei .e lo nu popkorka sepi'o lo spici sparkeli

.i le kompymacni na se verka lo dumbystedu .i .ei le rubrynekti sairviska cu kipygau le kotnypikci xance lo pokta

.i ko relka gi'e vatci le blinkenlaiti

Of course they don't, but if you choose to use a work of fiction as an illustration, it comes as a coherent package (assuming the author did a good job at worldbuilding) - you can't just take the bits you like and ignore the bits you don't and still have something coherent.

During the Butlerian Jihad, the people involved in it didn't know that ten thousand years of slavery would be what they are choosing, so we don't know how bad it actually was.

The way Jihad presented in the oblique references in books, anyway, it sounds a great deal like a fanatical fundamentalist religious movement to me. Those are not known for their fine reasoning.