Hacker News Reader

Comment by mark_l_watson | Hacker News Reader

I manage a deep learning team but I have some reservations about the technology. IMO deep learning is best for optimizing back end systems and not good for systems that ‘touch’ people: deciding to loan money, automated sentencing of criminals, targeted marketing from personal information, etc.

For me, the problems are lack of explainability and possible bias.

There are many great applications for deep learning and AI in general but some guard rails must be in place for public good.

View on HN

Being able to shrug and say “the algorithm did it,” is almost certainly seen as a killer feature for any authoritarian, soulless megacorp and others. Instead of having to take responsibility for decisions, they just point at the box they programmed to act a certain way, and blame it. Unless a lot more people understand GIGO, this is going to stick around.

Exactly, this is easily one of the scariest things to see happening. Examples like Google's recommendation algorithm pushing people towards more and more extreme content (or even bot created content) signals to me that we're headed down a very dark road.

Right now the big guys can put the blame on their underlings but blaming it on machines will be even better.

These systems will be the perfect "faceless bureaucracy". Nobody knows exactly why they are doing what they do and can't be held responsible but the people who deployed them will get the profits.

Until people figure out how to hack them for profit. An AI with permissions to spend money on the company's behalf is going to fuck up eventually (and eventually in a major way).

Then it's back to human bureaucracy.

The funny thing is that has already happened with automated traders and they still keep at it. They have gotten fooled before by misparsing twitter feeds and they keep at it. They have gotten some kid-glove reversals though.

The thing is that it just needs to mess up less than humans to be worth sticking with it or have the political inertia to be 'preferable' to humans bureaucracies making the decisions even if it is sub-optimal. Zero Tolerance in schools is a godawful policy but because it lets them cover their asses even when it results in them getting sued and losing due to wrongdoing by trying to avoid frivolous lawsuits which they would win it is unfortunately sticking around.

The whole reason bureaucracies proved useful over just fiefdoms is that constraining to rules worked better than leaving everything to the discretion. Even the infamous 'flower poetry' Chinese exams were a leap forward because it meant that anyone who could prove sufficient literacy could get government jobs instead of just those connected and offered a floor. Not a great one mind you but literacy is a pretty good baseline for 'capable of handling paperwork and worth giving a decent paying indoor job'.

Probably some will benefit but most people will have difficulty rectifying issues that aren't in their favor. Look at how difficult it can be to reopen a wrongfully closed PayPal account or just to get an explanation.

A 'Medusa' browser that will constanly spawn chaff:numerous instances of false data and meta-data, patent it, Google hates it, buys it.

..predictably, angry Abbot RMS/openSource commandeers 'Medusa' into 'gnudusa' github forks to 'nudeusa', 'goregon', 'gone', vpn networks p2p, 'blockchain' likely tossed in somewhere, conflates concurrent user data/meta-data into noise, mass outrage wide adoption undermines, saps, destroys walled gardens.

You make it sound like bureaucracies with permission to spend money don't fuck up all the time.

I think you have captured the essence of it. What I wonder though: have systems before been without bias? Is the DL/ML bias worse than the one we had before?

For ML systems, it's an engineering mistake to deploy a complex model when you don't have a simpler baseline (e.g. does this outperform a basic n-gram model?). Similarly, it's a strategic mistake to deploy a deep learning model without assessing the baseline of human performance (including bias).

I see the problem of inexplicability as less salient than (1) responsible, informed deployments of models, and (2) ongoing measurement (especially against a human baseline).

You can deploy explainable models without (1) and (2) and end up with a much, much worse result.

The important phrase there is engineering mistake.

Intelligibility and ongoing responsible measurement creates a performance metric, and a line of responsibility.

To many, especially if they receive large pay but are incompetent and/or face legal risks if found liable, these are significant benefits.

/depressing, I know...

But if the logic before (for, say, whether to loan people money) was some sort of flowchart or checklist or whatever, it may be bad, but it's inspectable, and so could be examined, evaluated, and changed. The DL/ML creates effectively uninspectable black boxes.

Reply on news.ycombinator.com