Hacker News Reader

Comment by wwwigham | Hacker News Reader

You could call it Luddite, but I think I'm much more comfortable with human judgement being involved. Especially since that also provides an obvious place for blame to lie, if such preventative enforcement is used extrajudiciously or without reasonable cause (there is a degree of tension between supervisors and workers - that's part of why unions exist). I can also trust to some degree that the supervisor will pull someone aside and engage in a conversation, as a fellow person - robosupervisor might not be so considerate without serious consideration by it's implementors and executors (especially if such consideration doesn't help the bottom line).

View on HN

Why do people jump so easily from "lets build an efficient monitoring system" to terminators? Why do you think that first action that will be taken in such a condition will be something bad for the employee in question? Might be a simple notification for the human supervisor to check the situation out or it could even be a notification for the employee: "don't walk there, you might fall in the bloody tank".

Because in terms of human relationships, there's such a thing as "too efficient". Compassion, mercy, discretion, joy, friendliness - they all exist in the inefficiencies of a system, they happen in places the ruthless process hasn't reach yet. They die when turned into code - whether on computers or in papers, as bureaucracy is nothing but software running on a distributed protein runtime.

People aren't worried about a system that can deliver a timely safety warning to an employee. People are worried about a system that can spot everything that looks like wrong or inefficient use of your time, everything that may perhaps lead to a problem but didn't yet, and count all that against you, and then keep counting until you cross the threshold of getting fired, and there's no one who cares or even can do anything to help you.

Systems are buggy, their inputs aren't reliable, they have no understanding - and it's all too easy to everyone in the command chain to excuse themselves from responsibility by saying "but the system says so", or "but the system doesn't let me!". Next time you're wasting hours dealing with a customer support clerk of a bank or a corporation, constantly hearing "the system won't let me do that", think that at least it's just some product you're paying for, and not your very livelihood.

> Systems are buggy, their inputs aren't reliable, they have no understanding

Unfortunately this gets less and less acknowledged by many of today's technicians and engineers (I'm a programmer myself). People need not necessarily focus that much only on the Kafka-like stories or on the Bentham-inspired panopticon (both of these being real problems, I agree), but instead they should also go back to (re-)read Stanislaw Lem's "The Cyberiad" [1] in order to see what unreliable and buggy systems can do to a society that relies too much on them.

[1] https://en.wikipedia.org/wiki/The_Cyberiad

Why do people jump so easily from "lets build an efficient monitoring system" to terminators?

Probably because humans (especially governments) have repeatedly proven their willingness to take innocent tech and use it nefariously. That's how you go from CCTV being installed to "catch a few crooks" to it being used to monitor citizens for social credit systems. Or people's innocent internet use being hoovered up by massive surveillance projects.

It doesn't seem that crazy for people to wonder at how initially innocent tech might be weaponised.

Because there are far too many working environments from America to Kazakhstan where that is the default action.

Also the psychology of the original panopticon applies. It's a source of stress to be watched when you can't watch back.

>Why do people jump so easily from "lets build an efficient monitoring system" to terminators?

Because every technology that can be effectively used to oppress people has been.

While you can't stop technological progress it's farcical to pretend that people trying to oppress people will not use all the technology at their disposal. You cannot get the good with the bad.

If history is any indication I'm betting many people will have to get hurt by <thing> before society forms a consensus that it isn't ok to use <thing> to hurt people. It took WW1 to convince practically everyone that chemical weapons weren't worthwhile). It took the holocaust to get society to agree that genocide in developed nations (note the qualifier) is not ok. I'm not going to guess who will go down in history as the reason we don't build skynet but someone will likely have to before we stop building skynets.

Reply on news.ycombinator.com