Hacker News Reader

Comment by userbinator | Hacker News Reader

If I look into my mirrors to make sure I am safe to change lanes, it dings me for distraction because my face is turned to look into my mirror.

WTF.

This seems like the real-world analogy of a similar trend I've noticed in software development where, to "improve code quality", humans are replaced with stupid tools that give false positives all the time, and to placate their warnings and satisfy the metrics of the management imposing them, writing code becomes perverted into trying to figure out how to stop the tool from complaining instead of focusing on the actual problem being solved.

View on HN

I am living in this hell at the moment. This situation originated from compliance box ticking and trickled down to a kafkaesque nightmare where you are punished for ancillary problems. This is of course while being hammered because delivery has declined to a snail’s pace because the numerous hoops that need to be jumped through and general cruddiness of the process get in the way.

But someone somewhere high up in the ivory tower ticked a box. I suspect the same here happened with Amazon and an insurer somewhere. Push the hell down to the bottom of the stack.

The funny thing in this case is the process holes are still so enormous that you can drive a tank on fire through it and straight into production. The net gain is zero and the outcome is overconfidence.

Taser the camera. "Dunno, hoss, didn't notice it wasn't working. "

Speak softly and carry a can of spray-paint. (Hairspray might work just as well and be more defensible.)

sure, so the AI knows you're driving and it can NEVER see your face.

Reminds me of something that happened a few years ago to me.

Company was primarily using Java for the backend. The Application Security team decided to start scanning all our deployable JARs and WARs with a static analysis tool. Well, immediately the tool started complaining about our libraries. For example, the official Elasticsearch library, for the version of the database we used, had multiple System.out.println calls. Well those were all flagged and the static analysis tool deemed our code unacceptable. We needed to use a logger, not stdout. So we had to go and take it up with AppSec. And explain, no thats not our code. Yes, it does run in production. No we can't use a different library.

The whole thing was exactly as you put it, a kafkaesque nightmare where I'm being punished for something I don't understand by people who don't understand. An no-one can tell me why.

Just want to point out that Kafka based his stories in part on his experience working in the (Hapsburg i think?) bureaucracy, in particular the idea of a faceless and impenetrable organization that can't be reasoned with an is full of catch-22s (which I haven't read). So I find it funny we call it Kafkaesque when Kafka was just referring to how bureaucracy is. We should call it bureaucratic and admit that is what all big orgs end up becoming.

Phew, this at least spot some real issue (in some cases System.out.println can make multi-threaded program to get "synchronized" on console acces).

At some point I came across a tool (I think it was white source) that was going through Github page of each library used and was checking if there are no bugs reported.

Obviously very often there were totally bogus tickets, support requests created as bugs, rants as bugs, duplicates of already fixed tickets, etc. so investigating all that crap was really hard.

Another problem was licencing. There are a lot of libraries that have dual licence, e.g. GPL and CDDL, obviously the tool was panicking that GPL is used, not noticing that there is a second "business-friendly" license.

Those static analysis tools are not bad as a concept, however ratio of false positives to real issues found is too large to make them truly useful unless one can invest a lot of resources into tuning them.

I think we're in a huge bubble in term of automation/AI these days, people are convinced they're very close to being able to automate most things while in reality we're faaaar away from even simple tasks. We're high on tech and the afterglow will hit hard

The seemingly very smart people designing these systems don't understand that their 1 in a billion edge case happens hundred of times every single day when you scale it to the world

We're in for a few decades of poorly implemented automation, to the degree many will feel they live in a dystopian nightmare. The arrogance of our management class to think society can be automated without a society wide analysis of what that automation's unexpected side-effects are going to be is equal to the colossal disaster we're going to see. I serious feel sorry for everyone that is going to live through this period. We're gonna be creating misery for a few decades, until the assholes on top figure out this needs massive pre-planning and the seat-of-the-pants methods being used now is just setting up for a planet-wide disaster.

can the driver just put a photo of himself in front of the camera and score total bonus for being always on attention. Or (using the Moscow street surveillance contractor recipe) if the system is smart to not be fooled by a photo - record one ideal day (or even just an hour) and put the smartphone with the video in front of the camera on repeated play.

Of course the most interesting is to use counter-AI to generate video of right behavior without repetition and with matching location/street view in case if the Amazon/Netradyne AI is smart enough to have geolocation based street view integration.

I suspect the people who understand AI well enough to fool it also know they can get more money not being a driver.

I think that deepfake tools will soon be available to everyday user just like filters in Instagram or basic video editing tools.

And even if we just limit ourselves to tech users - some companies have long been logging coders' Eclipse (mostly not in the 1st rate countries of course), and I suspect these days that may get fed to AI. And some day it will make into US and West Europe too.

You would be surprised how quickly people can learn how to fool something, without knowing exactly how it works.

Uber drivers used to look up time slots for big planes arriving at the airport. They would then all go offline on the app to trigger "surge pricing" and then all benefit from it.

No one knows exactly how the algorithm works, but they know how to fool it.

interesting aspect here is that otherwise honest people at some point of unjust pressure put on them start to allow themselves to cross into that gray zone, and without feeling shame it leads to that being done in the open inside those specific social groups affected and thus becomes the acceptable social norm there.

I think people are able to segment moral behaviours very effectively. Uber’S ride allocation system would be very far down the list of things I would feel bad about gaming.

> interesting aspect here is that otherwise honest people at some point of unjust pressure put on them start to allow themselves to cross into that gray zone, and without feeling shame it leads to that being done in the open inside those specific social groups affected and thus becomes the acceptable social norm there.

The thing to consider is that no actions are taking place in a vaccuum. The drivers gaming the system is not an isolated input that is occurring. Instead, there are a number of different inputs that are putting downwards pressure on the rates a driver gets to charge / how much money the driver actually takes home after costs, taxes, etc.

While it's not incorrect (or even unjust) to describe the behaviour as being potentially 'gray' (although the use of colour is a different matter, I understand the sentiment being expressed), by only highlighting and focusing on the least powerful members in the system, it unconsciously paints a picture that draws focus away from the other inputs into the system that have resulted in this situation occurring in the first place.

Yeah, that’s fair. Cargo cults got famous because they worked, despite the mocking way they are often reported.

What makes you think people who take a job driving are incapable of understanding things as well as you?

As soon as someone has developed that understanding, they get a chance for a massive pay rise and better working conditions.

While I acknowledge that some people will prefer to be a driver than to work in any of the places whose doors would open with this skill, I suspect this is a minority.

> can the driver just put a photo of himself in front of the camera and score total bonus for being always on attention. Or (using the Moscow street surveillance contractor recipe) if the system is smart to not be fooled by a photo -

Only if you know what the AI cam sees, if it's like a blackbox to you then I doubt there's way that you could do to fool it.

After partially obscuring the lenses by say an accidental drop of ketchup :

"Your camera doesn't get correct image"

"Why?! What is wrong?"

"See for yourself"

It's hard to believe or every single driver would get the same consequences. This reminds me of drivers who complain about speeding tickets right after the speed limit changed. Well, slow down before the speed limit changes and you'll be fine. Some people just have bad habits without realising.

This comment reminds me of a, thankfully small, subset of developers who believe their code is infallible and it must be the users fault if the application has bugs.

What the AI is attempting to solve is a complex problem. I’m willing to give the drivers here the benefit of the doubt ;)

This comment reminds me of a, thankfully small, subset of drivers who believe their driving is infallible and it must be the other drivers' fault if they're penalized. What the driver is attempting to do is a complex task. I’m willing to give the AI here the benefit of the doubt ;)

You clearly didn’t read the article then. The drivers are open to dialog. It’s the maintainers of the software and/or Amazon who are assuming the system is perfect.

I’ve worked on smart interfaces for cars by the way. My experience not only as a driver but also as a developer in this space tells me that AI described in the article needs further development and your presumptions in this discussion are BS.

You're missing the point. If the AI is consistently penalizing drivers for looking in the mirror, then it'll affect them all equally and not be unfair. If it treats some drivers worse than others, why didn't they mention that instead of leaving us to assume that there really are drivers who are never penalized for looking in the mirror?

TFA even points out drivers making excuses for not wearing a seatbelt. If they won't even take personal responsibility for their own illegal actions, of course their opinions on more subtle things like following distances aren't reliable.

> You're missing the point. If the AI is consistently penalizing drivers for looking in the mirror, then it'll affect them all equally and not be unfair. If it treats some drivers worse than others, why didn't they mention that instead of leaving us to assume that there really are drivers who are never penalized for looking in the mirror?

They did mention it: not every driver has the AI installed.

Also you’re assuming the bonuses are awarded like prizes to the top performers. So if everyone is penalised equally then the same people still get awards. My impression was those bonuses were awarded to anyone who passes specific milestones. So even if everyone is penalised equally it’s still unfair to everyone.

> TFA even points out drivers making excuses for not wearing a seatbelt. If they won't even take personal responsibility for their own illegal actions, of course their opinions on more subtle things like following distances aren't reliable.

Yes, that is a fair argument. I interpreted that in a more charitable way saying “they’re acknowledging some bad practices on their side so at least trying to meet the AI developers half way”. But I will admit your interpretation is just as, if not more so, plausible.

Even so, that doesn’t justify being ignored to by Amazon and the AI developers. Any new tech needs debugging in the field. And this is more so true with AI and complex problems like driving assistants than it is in most other software specialties.

That’s the part of the article that stinks the most and the original point I was making yesterday. I couldn’t give a rats ass who is to blame because the biggest fault lies with the company for not working with the users to ensure the software is performing correctly. And this is especially important when people’s income are being directly effected by the software too!

Yea not excusing Amazon ignoring their requests or blindly using AI results to decide compensation, or that it can encourage dangerous driving to get points, or that it's probably inaccurate. Overall awful, but might be a fun game if you didn't depend on the job for your livelihood.

“When I get my score each week, I ask my company to tell me what I did wrong,” the driver told Motherboard. “My [delivery company] will email Amazon and cc' me, and say, ‘Hey we have [drivers] who'd like to see the photos flagged as events, but they don't respond. There's no room for discussion around the possibility that maybe the camera's data isn't clean.”

You should interview at Netradyne. From your comment you seem like a good fit.

Automatically analysing safe traffic behaviour from a camera is a hard problem. From the article, it sounds like this system is incredibly naive and doesn't really understand real world traffic situations, so it ends up punishing people for doing the right thing while demanding they do the wrong thing (like not looking in the mirror), and on top of that, distracting them and adding extra stress.

It sounds like a system that needs to be banned or thoroughly regulated. Naive driver distraction systems like this do not make traffic safer. The company should prove that this system is safe just like autopilot manufacturers have to.

I'm talking about the unfairness, not safety. The article doesn't say that some drivers are allowed to do it while others aren't but still presents the whole thing as unfair.

Sure, but I'm talking about safety. And safety is also an important issue. And one that's at least as much subject to regulation. And regulating this infringement on driving safety would also help with the unfairness, even if it doesn't address it directly.

Of course the unfairness is also a valid concern, and I'd love to see them both addressed.

Reply on news.ycombinator.com