roenxi

Or the cult of Eliezer Yudkowsky aka Yud.

> And there is no doubt that, rightly or wrongly, this is what the article author means to accuse Rationality/EA of being.

There is substantial doubt, I don't think the article did accuse them of being any of those things. Obsessed, yes. Poorly reasoned, sure. Coercive and insular I am not seeing, and dangerous is a difficult reading to find. Whats the danger here, we accidentally maintain the status quo of the insufferable 2025? There is a totally unrelated firebombing invoked, I doubt the dude was in the sex cult and the US seems a bit CEOcidal right now. The most specific criticism I spotted is the article writer thinks they may, in a possible edge case where people take them seriously, hinder medical research. Which is not actually a rare policy position if you spend any time talking to people about what needs to happen to speed up medical research.

It also [0] defines a cult as great devotion to a person, idea, object, movement, or work (such as a film or book) or a small group of very devoted supporters or fans. And I put it to you that the article is clearly using the word more in that sense because it is quite difficult to fit the words coercive, insular, or dangerous to the AI safety people. The accusation in the article seems to be that they're too friendly and sexually inclusive. And if you want to communicate with them they appear to be active on Twitter and not shy.

> Well, technically military is run now by maga incel death cult, so yeah I give you that one. I forgot I was discussing on US forum.

Well, we're speaking in English. I was careful to phrase that in a way that applied to what I'm pretty sure is any English-speaking country. Anywhere that votes could put anyone they like in charge of the military for any reason, cult or no cult. It isn't technically relevant - or practically relevant in most cases. It is just a question of whether the cult has the numbers or not.

[0] https://www.merriam-webster.com/dictionary/cult

Because they don't get enough votes to control the military. If they put a convincing pitch in for why they should then they can. Being a cult doesn't rule them out.

> You do understand what a cult is?

Pretty much any religion and a few other organisations beside. Basically people who believe something strongly. What do you think a cult is, in the sense the article is using it?

I mean, this is a rather fun article to read but I'm not entirely sure about the thesis.

> AI Safety Is Mostly A Sex Cult.

> I don't think these people should make policy. (1/?)

One of the learnings from things like the whole Epstein saga has been there are a lot of sex cults in the policy making world and, yes, the people making policy generally shouldn't be. But I'm not sure this article actually comes up with a good explanation of why these particular people shouldn't be making policy. People having active sex lives isn't a disqualifying trait. Being in a cult doesn't disqualify people from being in politics. It is routine and expected that fields will have a lot of similar people in them (as I recall the overlap between security researchers and the furry community is unusual too, and that is hardly more important than justifying a passing comment).

It is more routine than the article writer seems to think for all the people in a small field to know each other and get on really well.

But the human would have to get extra hardware to do that. The AI isn't bringing in any resources it doesn't already have access to.

Well, yes. I do. Why do you want to consider mining Monero with a shitty custom CPU? That's a bit of a non-sequitur and I don't see the relevance to the thread. We've been in furious agreement on that point for about 4 posts now and I'm not sure where you're trying to go with it.

I'm going to reiterate that you are the one who started calling a CPU an ASIC, I'm was just trying to match your language to be friendly. A CPU is technically as ASIC for a certain class of computable functions, so if you want to call it that I'm happy to as well. If you don't want to call it an ASIC we don't have to, we can just call it a CPU.

It's an interesting puzzle, isn't it. On the one hand, the AIs are no good at playing Chess.

However, on the other hand, if you ask an AI to win a game of chess it has all the tools on hand to compete at the same level as Stockfish - it can re-implement an engine and even probably has a GPU on hand to train its own neural nets.

So should we say that the AI can play chess well, or that it cannot?

That sounds implausible. For a premium brand the marginal unit should be comfortably profitable, so they will have the resources to saturate the market at exponential speed purely from their unit economics. The somewhat theatrical lines outside the store aside, Apple has never had a strategic problem meeting the demand for Apple computers.

It seems a lot more likely that you get executives who come in and believe that narratives beat reality, there are a lot of people like that in the world. They are the sort who will abandon the reality of quality because that costs money then get confused when the narrative catches up. Or you get an idiot who is data-driven and detects that there is no immediate change in sales when they compromise on quality so they conclude that quality isn't valued by consumers. No malice, just a lack of strategic thinking.

You're the one who called a CPU an ASIC. Seems pretty general purpose, it's a Turing machine simulator.

And hot ice cubes are a thing. They've made ice at up to 650K if the wiki phase diagram is any indicator https://en.wikipedia.org/wiki/File:Phase_diagram_of_water.sv...

Oh yes, sorry. General purpose. I edited the comment.

> Like Monero but easy to mine is Monero. A Monero mining ASIC is called a CPU, by design.

That is the "does nothing" argument. But compared to Bitcoin, having a general-purpose [0] ASIC seems like it'd make the network less secure - if someone wants to run something like a 51% attack against Bitcoin they first have to buy a ridiculous amount of specialist Bitcoin-only hardware that will potentially lose all its value after a successful attack. Also probably be a lot easier to track who did it.

Monero the equilibriums are less obvious. Does that matter? No, I don't think so, I still own a certain amount. But it isn't a helpful axis to experiment on and I don't see where the upside is meant to come from vs just going with sha.

[0] Edited, was special-purpose.

It is a bit of a leap, but quite possibly an oblique reference to Voltaire's Candide. He spent some time dismantling the idea that this was the best of all possible worlds (the phrase apparently started with Leibniz) and the Lisbon earthquake featured prominently.

Yeah but their strategy on that seems kinda dumb to me because it either does nothing or has a negative effect on the ecosystem. If someone wants to mine bitcoin on their own CPU they can, and if my PC has enough hash power to find a coin on a blockchain then let me tell you, that chain has real problems and is not a serious player. That amount of computing power doesn't justify paying any interesting amount of money.

It isn't the biggest technical concern and at least they have principles, but I expect something that looks like Monero + easy to mine will eventually appear and turn out to have an advantage in the real world. Although it is hard to predict because it isn't completely clear what (if any) impact the algorithm difficulty has in practice. But if there are big datacentre mining a cryptocurrency with dedicated hardware then that means there are people with skin in the game who want it to succeed. Seems like a better equilibrium than whatever the Monero people are aiming for.

The article is exploring a broad topic but one area that seems to be overlooked is governments making privacy illegal or actively working to undermine it. The major privacy risk right now is things like social media age verification laws which, in principle, probably make pseudonymous sites like HN illegal or on the path to being illegal in many jurisdictions.

It is hard to maintain privacy when a law enforcement body insists that it isn't allowed. There are also indications that companies quickly become too enmeshed with intelligence services to be able to create properly private options even if tactically they might have an interest. Companies like Telegram stand out as looking difficult to run in practice because of the pressure from various snoops.

If it is legal to create private options then they will always exist because the people who care about specific aspects of privacy can self-serve.

> The point being: humans seem to me to be the weak link here.

I tend to agree but it is hard to shake the feeling that there is a larger system in play that the humans are just a component of. And that system is making the decisions.

Historically that whole thought was just a philosophical curio because the decision making parts of the system had to be powered by humans. But what we're discovering as AI improves is either we've hit AGI or humans are actually incapable of performing any act that demonstrates intelligence or autonomy.

As we build systems where the drive and decision making stems from computers, it does seem that we will have to revisit the concept of humans being the problem.

> ...from a checking account. Banks ...

We've got Schrodinger's money here! One minute I can go down to the bar and create as much money as I like with the bartender. Then the next we suddenly discover that yes we need a bank and yes there all of a sudden a checking account is involved and no a bar tab isn't a legally recognised form of money.

> I'm not Nvidia. I did just sign a purchase order with a builder...

Again though, we discover that in fact the person in control of the part where the actual money was created was a bank. You can't just head off with your builder and create money - otherwise you'd be stupid to stop until you unseat whoever was the richest fellow in the world this morning.

> Nvidia causes the creation of new M1 through banks.

And the banks make another appearance!

---

I put it to you that Nvidia can't, in fact, create new money. You'd never get that line of argument past a tax collector.

> Fun fact, you don't need a banking license to issue traveler's checks. And traveler's checks are counted in M1 in the U.S.

And I'll add in postcript that I know nothing about travellers checks in the US, but I'm quietly confident it'd turn out to be illegal to go around creating vast amounts of new money there too, on the basis that people tend to work for a living.

> Yes.

How? And what is your theory why people pay taxes from their labour earnings instead of just creating new money to pay Just-In-Time?

> A lot, however, will be created through loans (new M1)

Isn't M1 is currency and overnight deposits? I'm pretty sure you're just making things up here; it sounds highly illegal for Nvidia to create new M1. Unless they've taken out a banking license when I wasn't looking.

> You can pay your taxes out of a checking account using money a bank created out of thin air that morning that the Fed won't even learn about until the evening.

Ditto prior. Nvidia isn't allowed to do that.

That's not sexist though, it is just equivalent to noting she's a woman (like calling her "her" and "she") because we can reasonably offer the same criticism about a man. A man can have an extremely arrogant demeanour instead of the controlled gait and bearing proper to the stronger sex. And adjusting for language drift people do make that sort of complaint of men, I remember one criticism of Biden being that he walked in a frail way and fell over a lot - literally complaints about his gait and bearing giving a bad impression. Not a very strong criticism but he was male and it was made.

We might reasonably infer that she got called arrogant for doing something that wouldn't be considered arrogance in a man, the article has unhelpfully elided that detail (maybe the original author did as well). But we've also got to admit she might have actually been unusually arrogant, that does happen from time to time. If I were in charge of a military I'd be standing around even more arrogant than I already am.

pjc50 offered a good example of something that was actually gendered - women not being able to own property in their own right for whatever unfair reason. The quote isn't that sort of out-and-obvious sexism. Said quote could almost be modern political whinging.

PS I'll add in an acknowledgement that words did mean different things back then so possibly at the time the only reasonable reading was with an "[as with all women]" on the end, but the fact is that as written this quote is making specific accusations of arrogance and arbitraryness that are not contained to either gender and are bad in all leaders, female or otherwise. He didn't write "women are all arrogant" or "women are all arbitrary". If anything the thrust of the quote is most women weren't either of those things. It doesn't support the idea of double standards or ludicrous criticism that would never be made of a man.

If we attempted to pay our taxes by creating this new money, do you think we would succeed? Because I suspect that when we get to the tax office we will discover that what you're calling money is not actually money in the practical sense.

Or I suppose I should spend more time with an open bar tab come tax time.

The monetary aggregates are things that are theoretically equivalent to creating money, but I'm going to challenge whether it is proper to compare M4 activity to the fed balance sheet. One of those entities is involved in actual money creation. One of them is hyper-privileged money creation that is more impactful and less comparable to other forms.

> not on the article to triple its length to prove every minor claim.

Yes but that is my point. They're making the article longer by including evidence that undermines its claims. They should have either made the claim and let it stand without evidence, or they should have aligned what they said to the evidence they felt was worth including.

If they're going to claim gendered criticism then hold up generic criticism as the best example of criticism it confuses the picture. Assuming that there was gendered criticism happening somewhere off to the side out of sight, was that the decisive criticism? Or was it the criticisms that the article writer felt were important enough to include in the article?

Because the impression the article is giving is that people weren't complaining about her gender, but the fact she was behaving arbitrarily and arrogantly. That is good reason not to put a man in charge too - if you see a man acting arbitrarily and arrogantly it is also a mistake to put him in charge of anything and he would be criticised for it. I do that all the time, there are a lot of unprincipled men in politics who should be called out for their hubris. And they are.

Now it may be that what she was doing wouldn't be considered arrogance and hubris in a man (we don't know exactly because the article is short) but "double standards" and "sound ludicrous if aimed at a man" are simply not true. We hold men to the same standard and make the same criticisms of them.

My point is the article claimed it would show evidence of something when in fact it did not. It should either not have bothered to include evidence or found better evidence

It's not a handwave - the article doesn't offer anything to handwave away. The article says that she was offered specifically gendered criticism, then quotes generic criticism.

I'm sure being a woman counted against her, bit if the article is going to put up a quote as evidence of sexism then it shouldn't be a passage that can be realistically adapted to any politician ever. If the article had a passage that says "we can't let this woman be king, she can't even own property!" that would be different. But the quote in the article doesn't sound ludicrous when applied to a man. It sounds like something that is routinely said of men in positions of power.

It could even keep the superficial gendering if you wanted it to, there is a lot of argument that men in positions of power should be exhibiting positive masculine bearing (stoicism, accountability, honour, etc, etc) that politicians routinely fail at to great public outcry.

> The double standards are evident when we consider that all of this would sound ludicrous if aimed at a man who had been proclaimed king...

I don't know about that, the quoted criticisms sound a lot like those that would be levelled at a king too. The issue is more that nobody expects a man to care, even a little bit, if someone accuses him of being crude and arbitrary in his search for power. Obviously it is gendered, but the actual complaints are generic. Anyone with serious power has to bat through things like that.

I can give an example by swapping in Trump. The criticism holds up with a gender swap, adjusted for the fact that by default we expect a man to be in the office so it isn't explicitly gendered. This sounds like stock criticism of him that might turn up in any political conversation.

[Donald Trump] at once put on an extremely arrogant demeanour instead of the modest gait and bearing proper to --the gentle sex-- [his office], began to walk and speak and do all things more stiffly and haughtily than [he] had been wont … [he] arranged everything as [he himself] thought fit and according to [his] own arbitrary will.

> 'But Cisco did it too!' is just whataboutism.

A whataboutism that makes a legitimate point. It isn't reasonable to dismiss something just because a person makes a comparison. It's valid to consider that Apple might have been applying inconsistent standards and unfairly targeting Zoom for some reason.

I doubt they were being unfair but it is a bad practice to dismiss an argument because someone has the temerity to expect consistent standards. The threat of Apple arbitrarily removing apps based on unreliable reasoning is concerning.

> It absolutely is. Similar to the way banks create money [1]. The commitments support credit that wouldn't exist without it.

Making a loan/offering credit isn't automatically money creation - the amount of money in the system before and after the loan might be the same. Haven't been following Nvidia all that closely, but it seems a little bit unlikely that they're a commercial bank. Financial chicanery they may be doing but offering deposit accounts would be new territory. The loan has to be made in a very particular way for it to be money creation (notably, in a way that creates new money), and it should be illegal for most people to do that otherwise we'd all be printing our own money instead of the printing being directed to wealthy asset owners first and foremost.

> Nestled in the lush green mountains of Central Mexico, next to a sparsely traveled gravel road...

The economics of this are deliciously surreal and in many ways slightly romantic. You have this dingy little shack-like construction out in the "lush green mountains", run by these lunatics of a Mexican persuasion, coordinating billions of dollars of international financial data in a reliable high-trust manner. I encourage everyone to look at the photo in the article and imagine a fat dude in a singlet sleeping on a hammock because that seems to be the vibe of the place.

This is a real case of the truth being stranger than fiction. If I were reading about this sort of thing in a novel in the pre-crypto era it'd be a big building in a city or something. Maybe some dudes in suits. Sealed roads.

Turns out it's going to be closer to an unwashed hermit sitting in their mountain cave spending the days meditating, fighting off snakes and tending to the server racks. Voila, high finance.

How do you expect their material conditions to improve if nobody attempts to get more stuff to them?

I doubt the "HN crowd" was about to do anything, but it'd be better if someone saw some sort of market opportunity here. It sounds pretty miserable not being able to get to a well paid job. I personally appreciate all the people who lent me money and sold me cars. It enabled a lot of income for me, over a lifetime.

> Millions of Hong Kongers were determined to keep the freedom they had enjoyed. The CCP has won, at least for now.

Fundamentally, I just don't see this frame as being a path to long term success for HK. If it is them v. the CCP: the CCP is alert to the political risk of HK, outnumbers them, controls the local military, has better propaganda and has more of a claim to success given that the British Empire has failed, the US Not-An-Empire seems to be struggling and morally bankrupt. At least the CCP has been delivering visible improvements in prosperity over the last few decades.

It is a tragedy to lose the jewel of Asia that was old school Hong Kong. But they really need a frame that acknowledges a bit of the present political realities - there'll need to be cooperation with major factions in mainland China. Probably within the CCP. I assume there is a liberalising faction of some sort.

There seems to be a bit of a gap vis a vis solutions in the article - someone could set the system up so it doesn't incentivise tanking. The punishment for tanking should be that the teams performed poorly for no compensation. It's all very well to set up a system to encourage poor behaviour then shame the people who behave rationally. But it is much more sustainable in the long term to make good behaviour also strategically optimal.

> Post-Hinkie, the league experimented with variations of the lottery, but none of them could deter teams from losing on purpose.

I struggle to believe they've tried everything unless they're also talking about players betting against themselves (which probably is impossible to overcome without bans). Incentives aren't easy things but there has to be some system where winning > losing.

On that topic we're actually pretty lucky - in the real world, the winning strategy over long time-frames is almost always talking, prosperity and cooperation. Not obvious why the powers that be are so keen to involve other strategies that routinely deliver worse material and moral results.

It interesting because the people at the CIA probably could be fairly described as "liberal, non-violent, and honorable". That sort of thing is heavily into banality of evil territory - lovely people, just a bit thoughtless about the effects they were having on huge tracts of the world.

With those types the best you can hope for is at least they act rationally and effectively. It gets really sad when they do tremendous damage for reasons that make little sense, even in context.