Comment by petsfed | Hacker News Reader

Comment by petsfed | Hacker News Reader

Parent Full thread

You and I are in agreement that the surveillance needs to stop, but I think we differ on how to explain the problem. My explanation follows, but note that its not directed at you.

At its peak, the KGB employed ~500,000 people directly, with untold more employed as informants.

The FBI currently employs ~35,000 people. What if I told you that the FBI could reach the KGB's peak level of reach, without meaningfully increasing its headcount? Would that make a difference?

The technology takes away the cost of the surveillance, which used to be the guardrail. That fundamentally changes the "political" calculus.

The fact that computers in 1945 were prohibitively expensive and required industrial logistics has literally zero bearing on the fact that today most of us have several on our person at all times. Nobody denies that changes to computer manufacturing technologies fundamentally changed the role the computer has in our daily lives. Certainly, it was theoretically possible to put a computer in every household in 1945, but we lacked the "political" will to do so. It does not follow that because historically computers were not a thing in society, we should not adjust our habits, morals, policies, etc today to account for the new landscape.

So why is there always somebody saying "it was always technically possible to [insert dystopian nightmare], and we didn't need special considerations then, so we don't need them now!"?

This is the correct take. As the cost to do a bad thing decreases, the amount of political will society needs to exert to do that bad thing decreases as well.

In fact, if that cost gets low enough, eventually society needs to start exerting political will just to avoid doing the bad thing. And this does look to be where we're headed with at least some of the knock-on effects of AI. (Though many of the knock-on effects of AI will be wildly positive.)

> eventually society needs to start exerting political will just to avoid doing the bad thing

we hit that point years ago. the renewal of the Patriot Act comes to mind.

all they needed to do was to let it expire. literally do nothing.

> The FBI currently employs ~35,000 people. What if I told you that the FBI could reach the KGB's peak level of reach,

You are, if anything, underselling the point. AI will allow a future where every person will have their very own agent following them.

Or even worse, as there are multiple private addtech companies doing surveillance, and domestic and foreign intelligence agencies, so you might have a dozen AI agents on your personal case.

Aren't we there already? Sure, perhaps the fidelity is a bit grainy, but that's not going to remain as such for long. With the amount of data (for purchase) on the free market, the FBI, KGB, MoSS (China), etc. all have a solid starting foundation, and then they simply add their own layer upon layer on top.

I read "The Age of Surveillance Capitalism" a couple of years ago and she was frighteningly spot on.

https://en.wikipedia.org/wiki/The_Age_of_Surveillance_Capita...

The solution is sousveillance.

If they watch us, we watch them.

Cost is one factor, but so is visibility. If we replaced humans following people around with cheap human-sized robots following people around, it would still be noticeable if everybody had a robot following them around.

Instead we track people passively, often with privately owned personal devices (cell phones, ring doorbells) so the tracking ability has become pervasive without any of the overt signs of a police state.

I think if you bring up a dystopian nightmare, it assumes someone in power acting in bad faith. If their power is great enough, like maybe a government intelligence agency, it doesn't need things like due process, etc., to do what it wants to do. For example, Joe McCarthy & J. Edgar Hoover didn't need the evidence that could have been produced by AI-aided mass surveillance to justify getting people people who opposed his political agenda blackballed from Hollywood, jailed, fired from their jobs, etc.

If everyone involved is acting in good faith, at least ostensibly, there are checks and balances, like due process. It's a fine line and doesn't justify the existence of mass spying, but I think it is an important distinction in this discussion & I think is a valuable lesson for us. We have to push back when the FBI pushes forward. I don't have much faith after what happened to Snowden and the reaction to his whistleblowing though.

I think it is greyer than this.

Joe McCarthy and J. Edgar Hoover, distasteful as they are, I believe acted in what they would claim is good faith. The issue isn't that someone is a bad actor. It is that they believe they are a good actor and are busy stripping away others' rights in their pursuit.

Very nearly everybody -- even bad people -- consider themselves one of the "good guys".

Maybe in their minds they thought the end justified the means, but there's no way anyone, even themselves, mistakes those means for good faith.

Sure. the question was whether AI will continue to enable mass spying and making the affordability of mass spying a tool humans rely on.

They will. It will. We will see that erosion.

That's what this article argues though. Even with good faith acting this would be a disaster. Imagine any time you did something against the law you got fined. The second you drive your unregistered car off your driveway (presumably to re register) you are immediately fined. There may be "due process" cause you DID break the law, but there is no contextual thought behind the massive ticket machine.

Our laws are not built to have the level of enforcement that AI could achieve.

Interestingly enough, in some places, automation like that, like red light cameras, even after getting installed, were later prohibited. NJ has discussed laws around protecting NJ drivers' privacy from other states' red light cameras, too. It's important not to be complacent. You can imagine literally anything, but action is required to actually change things.

> I think if you bring up a dystopian nightmare, it assumes someone in power acting in bad faith.

I don't agree with this. I think it's entirely possible for a dystopian nightmare to happen without anyone acting in bad faith at all.

"The road to hell is paved with good intentions" is a common phrase for a reason.

It may be a common phrase, but I’ve never seen such a road myself. Mostly bad outcomes are preceded by bad intentions, or lazy ones, or selfish ones.

I’d be interested in a couple of examples, if anyone has good ones, but I’m pretty sure that if we put stuff like 737MAX MCAS, the Texas power grid fiasco, etc the count of badly paved roads would be greater.

The people that support the War on drugs would say it's with good intentions. But it has put a lot of people in jail where it neither benefits them nor people around them, nor society at large. In many cases leading to worse outcomes for affected communities.

War on terror is not without adverse side effects either.

All of the great wars of the 20th century are perfect examples. WW1 started when an assassin with Serbian state backing assassinated the heir (to an 80+ year old Emperor) of Austria-Hungary. Austria-Hungary demands Serbia effectively let them carry out an investigation and enact consequences at their discretion. Austria-Hungary refuses. So Serbia invades them. This causes Serbia's ally Russia to move in on behalf of Serbia. This causes Austria-Hungary's ally Germany to move in on behalf of Austria-Hungary. This resulted in Russia's ally France joining in on the war, and so on.

Soon enough you had Brits killing Germans because a Serbian assassinated an Austro-Hungarian royal. The most messed up thing of all though is that everybody had a very strong pretext of 'just' behavior on their side. It was just like an avalanche of justice that resulted in tens of millions of dying and things not only failing to get better, but getting much much worse.

Since the winners won, and the losers lost, the winners must be right. So they decided to brutally punish the losers for being wrong, Germany among them in this case. The consequences imposed on Germany were extreme to the point that the entire state was bankrupted and driven into hyperinflation and complete collapse. And this set the stage for a young vegetarian from Austria with an knack for rhetoric and riling up crowds to start using the deprivation the state was forced to endure to rally increasingly large numbers of followers to his wacky cause. He was soon to set Germany on a path to proving that WW1 was not only NOT the war to end all wars, but rather was just the warm-up act for what was really about to happen.

The other point is when technologies essentially make redundant an entire class of people then those people are more easily repressed.

In this case, it could be the entire US populace that is not part of the surveillance engine.

Reply on news.ycombinator.com