Don't Build Mindreading | Hacker News Reader

Don't Build Mindreading

lesswrong.com

21 pointspaulpauper14 comments

it's called less wrong because they're still always wrong

Lesswrong might be the only site on the internet where "mass-deployed BCI telepathy" is considered a more immediate threat to social cohesion than social media, propaganda or advertising.

It's a shame that today's "technologists" are eager to ignore the political axis of technology if it means they get to talk about ESP or Von Neumann machines. It's like I'm watching the field progress in reverse, transitioning from a crowd of self-righteous and skeptical consumers to a mystic and eschatological identity that eschews understanding.

> It's a shame that today's "technologists" are eager to ignore the political axis of technology if it means they get to talk about ESP

Your comment reads to me like a criticism of lesswrong, but the linked article is entirely about the political axis of a potential technology. Am I misunderstanding your comment?

Their argument against BCI is that it will be used in government-mandated purity tests to forcibly rally their subjects. But that's a political objective, and one that you don't need mind control or ESP to achieve. We already have extremely effective social controls in the form of Twitter/X, daytime television and mass advertisement. What use is mind reading in a world where you already control what most people think?

Look at the Patriot Act, or any other military censor you choose. Same concept - we need to peek into people's private communications to determine if they're a terrorist! We need to scan their photo gallery to prevent them from abusing children! All are means to the same ends, the state wants to enforce absolute loyalty by sucking out any room for protest. Throughout history you can very easily ascertain that authoritarianism isn't restrained by a lack of mind control, for every life Oskar Schindler saved there's another 1,000 bodies buried in Babi Yar. The fact that ESP might have prevented those 1,000 lives from being saved somewhat glosses over the millions that were ostensibly genocided without using psychic domination.

> Their argument against BCI is that it will be used in government-mandated purity tests to forcibly rally their subjects. But that's a political objective

Then why did you start this thread by complaining that technologists ignore the politics of technologies?

Because they are. This is a flight-of-fancy, a science-fiction diversion that has no relevance to any real world politics from the last century.

Their "Nazis with a mind reader" scenario explains it perfectly; only a small minority of people were able to lie well enough to escape a regime with 1940s technology. Millions of other people were killed and the Nazis did not need a mind reader to find them. A modern authoritarian state with a mind reader is barely any scarier than a modern authoritarian state without one.

> It's a shame that today's "technologists" are eager to ignore the political axis of technology if it means they get to talk about ESP or Von Neumann machines. It's like I'm watching the field progress in reverse, transitioning from a crowd of self-righteous and skeptical consumers to a mystic and eschatological identity that eschews understanding.

My brother in Christ, the call is coming from inside the house.

Reread my comments, I don't know how you're still confused about this. They are manufacturing a fictional scenario that does not significantly change the power imbalance of an authoritarian state, today or in their hypothetical example.

Today and in the Nazi era, there were functional social controls that served the same purpose and achieved, very clearly, dangerous results. No mind reading was necessary.

Yep, that is what every collaborator tells themselves to justify their actions. “Oh, it’s not like I made any difference, they were already going to kill all those people.”

This is like reading an argument against multicellularity. The evil neurons are enslaving all the rest of us. Mind reading tech would obviously just be a new kind of language. Whatever state comes after would be unpredictable and emergent. It's not nazis with brain scanners.

> It's not nazis with brain scanners.

Why not? Where do you suppose the nazis would go, once they got ahold of some brain scanners?

To modify the scanner to lie? Authoritarian states don't want the truth, it threatens their totalitarian ambitions.

You are making the same mistake as the author, gesturing towards the Nazis as if they were hamstrung by ethics and due process. They were not, they summarily executed their opponents without a chance for appeal. This is not the type of government that is systemically concerning itself with accuracy or collateral damage. The Wehrmacht was much more interested in the shock-and-awe style of terrorism that the V1 and V2 weapons afforded, there was no point in scrutinizing the German population further because their biggest interior threats had already been pogromed.

You’re probably right. The nazis, if given the opportunity to use this technology to root out dissidents, would probably just say “no thanks, we’re already so good at killing, we don’t need to bother with this new technology.”

You are mixing up what authoritarian leaders want to know with what they want the public to know.

Reply on news.ycombinator.com