Hacker News Reader

Comment by akerl_ | Hacker News Reader

This seems like the kind of end-run around the system that tech folks tend to think is a clever life hack, only to find out that legal/regulatory bodies aren't fooled. Like trying to convince the judge that you weren't speeding because if you set your frame of reference inside the car, it was the road that was speeding, backwards!

I only hope that the folks hand-rolling this onto their cars don't injure too many people in their attempt to "disrupt" cars.

View on HN

This isn't some "clever trick". It is genuinely not illegal to release a bunch of code open source on github.

The Government asked him to not sell his product, so he complied with the government's request.

It is an ethically shitty move, though. The software is not properly tested. People will run it thinking it's legit and probably some will get hurt or die. The law doesn't matter.

It's ethically shitty to release open source software now? People who pull down some source from Github that has 4 commits, thinking it's legit and rely on it to drive your car are probably not too many. Those that do, are generally hard to protect from themselves in any area of life.

7 years ago I completed a PhD developing the first control software for quadrotor drones. I mention this only to give a sense of time. Now every idiot with an arduino and a github account can crash one into my family at the park. Do you think the adoption of shitty self driving technology by unequipped amateurs will follow a different path?

The cost of both entry and failure for hacking on quadcopter vs car is orders of magnitude apart?

Unless you know something about the upcoming price of cars that you'd like to share ;).

Really?

In 5 years how many cars will be equipped with the basal level of sensors that make these sorts of hacks possible? My guess is more people will have compatible cars in their house than qudrotors....

Cost and chance of failure is poorly (or not at all) evaluated by the kinds of dangerous beginners likely to crash their car-pilot into a bridge.

The point wasn't about the ubiquity of the feature set in cars but rather the cost of bricking one/crashing one being well beyond what it is for a quadcopter. And the point still stands.

The people who are out there with quadcopters are the same people who would be doing these hacks at home on cars; some hobbyist/tinkerer subset of nerd.

I don't share your low opinion of their threat evaluation functions either.

Fair enough. Agree to disagree then.

I find it plausible that self driving cars with the same software can perfectly understand each other via predefined behaviour patterns.

If you follow that premise, it becomes less safe to drive manually.

So, under this premise, the modus operandi is to be automatic pilot instead of manual where perhaps manual driving is even illegal because it is less predictable.

The question wouldn't be, if this change happens; its when. Where's the turning point? One could argue its up to our politicians to decide. I'd be interested in data from other trends such as smoking. When was the turning point there, and what was the "market penetration" of smokers vs non-smokers or proponents and opposers?

So someone is an "idiot" if they hack drone code without a doctorate degree?

You should have written in code to keep the things from crashing into buildings and people.

> People will run it thinking it's legit

That might be believable if they didn't explicitly warn you and you didn't have to flash the phone yourself.

It insulates him from the government, not civil liability. If someone uses this and gets killed, he can still be sued and have to defend himself out of his own pockets.

Well, IANAL but wouldn't the license of the software protect him from that:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Him, maybe. Let's wait until it is tested. But it certainly won't insulate johnny github from liability

This is more of a disclaimer which won't stand up in a court of law. If he released the software and understood the risk that someone could use it and get into an accident and get killed, then the responsibility is on him.

Also, since this isn't released by his company, he alone will be personally responsible without any of the laws that would normally shield a business owner from repercussions of something like this.

This is similar to someone releasing malware. The government doesn't just go after the guy who downloaded it and used to get a bunch of credit card numbers, they go after the person who actually wrote the software:

MegalodonHTTP Author Arrested in December RAT Raid - http://www.securityweek.com/alleged-author-megalodonhttp-mal...

Ransomware Author "Pornopoker" Arrested in Russia - https://www.bleepingcomputer.com/news/security/ransomware-au...

Blackhole Exploit Kit Author Sentenced to Prison - http://www.securityweek.com/blackhole-exploit-kit-author-sen...

To say he's taking a huge risk is an understatement.

He still seems to be operating a "data network", to collect driving data for further optimization. I think that part makes the "selling" moot, given that his business is based on input from this "free" app...

> The Government asked him to not sell his product,

Well, the Government didn't that, they asked proof that it was safe before selling it https://es.scribd.com/document/329218929/2016-10-27-Special-...

They specifically used the words "delay selling your product", which is what he did.

To give an example of another field, the OpenAPS project for diabetes seems to be walking a very narrow tight rope in their actions and statements in order to not be regulated by the FDA.

https://openaps.org

This doesn't seem to be a narrow rope at all. Code is speech, as determined by the Ninth Circuit Court of Appeals in Bernstein v. United States. The government cannot prevent you from publishing very detailed descriptions on how a regulated device would operate, which is all OpenAPS is.

It didn't really strike me as an end-run but more like his interest was in solving the self driving car problem and when faced with solving the regulatory problem he said "no thanks" and walked away.

It is also a reason that as engineer (real degree, not self named), I support software being regulated just like everything else, not only on life critical situations.

Thankfully at least EULAs are void in most European countries and companies can be accounted for.

Sadly, despite 50 years of work on it (and there are still lot ot work on it), we have no idea on what to regulate. Which is the culprit here.

How would that work practically? I am sometimes for regulation but I cannot fathom what it would look like.

ANAE, but one form it could take is via a government or independent expert body with a mandate to inspect and certify code in some meaningful way before it reaches the consumer.

All code? I'm pretty sure you just ground the industry to an halt, if even Flappy Bird must be inspected and certified.

If it's just code used in critical situation, then you're not really talking about the same thing as pjmlp.

Self-driving car code is a critical situation, surely? Isn't that what we were talking about?

The code is likely useful in many other ways than just self-driving cars.

Ancestor in this thread said:

> I support software being regulated just like everything else, not only on life critical situations.

Maybe there is a line that is "critical", but not "life critical", but ancestor's statement was pretty damn broad.

People seem to be under the impression that regulation inevitably means "someone has to review/audit every single change that was made to any aspect of a product". But in practice there can be all kinds of provisions, from the weak to the strong.

A weak provision may be that at least a minimum of development and testing has to take place, with requirements on record-keeping and naming of responsible persons: For a simple device, or a simple software, this might take the form of a checklist or a questionnaire which you have to take before uploading code to the App-Store.

Yes, this seems very straightforward to fake, but at least then you have deliberately cheated a system designed to catch errors, and not "just written shitty code" (or built a flimsy mechanical device... when applying this scheme to non-software world). Which, I think, would already help a lot to make people accountable for their products/inventions/...

{for, example: with CE certifications, you, as a producer only state that you have built your products according to some harmonized standards, but the conformance itself is not tested by a third party}

With increasing security/privacy requirements, the mandated measures can could then be ramped up. Maybe scaled by permissions of an application (accesses otherwise protected data?), the ability to process sensitive date or maybe number of users?

The extremes measure would be a full line-by-line review on every change, when software is life-sustaining or a fault may have catastrophic (death of people, like running a pacemaker or steering a huge plane).

As a producer, CE certification is very disconcerting. I started out with the mindset of "how can I be certain that I'm compliant?", and the best answer is that it simply doesn't work like that. It's more of a question of whether you have enough paperwork to cover yourself in the (pretty unlikely) event of litigation.

There's some testing you can do, and you can sort of guess which product categories fit you exactly and apply those, but if your product is really innovative this is vague. You end up just putting a declaration of conformity in the box and a sticker on the product and hoping for the best.

I liked the CE process and thought it was pretty clear. You need to identify hazards and classify them according to probability of occurance and severity, then take appropriate measures to reduce the risk of harm. It's all spelled out pretty well without being a precise code (like NEC or UL 508A or something), so it's flexible enough to apply to whatever domain you're in.

This is how medical devices are regulated. The FDA doesn't micro manage or guide your development. They just say "you must have a quality management system spelled out and stick to it. However it's written you must obey it. "

There's a little more to it. There's some CFR standards that you must adhere to (it's been 12 years, so I don't recall the CFR part number off the top of my head).

We have all our code inspected and pen tested under different regulations and certification rules; if you would take that even a bit broader, no software would be written anymore; all programmers would be busy checking code.

And this software is not 'that' critical (financial). I would actually agree that life or dead software should be checked line by line by experts, software and well described processes. I would even insist that critical software that can be formally verified should be formally verified. But for a broader range of software this is not possible and well, I don't think it should be needed for innocent software like, say, games.

What could work for non critical software is having to pay for bugs if software is faulty based on the severity of the bug. I would be very much against the kind of liability that involves again kind of indirect damages, so the maximum of damage is limited to what you paid. Most people do not want that but you would want to have some or all money back in case of neglect (bugs that do not properly get fixed) without forgoing your license. That will kill companies (if turned back the clock with this rule, MS + Oracle notably) but then again, they won't cut corners like what happens now.

This may very well be the future text-book example of how to not handle life critical systems and software, unfortunately it may well take people being killed. At this point we have no idea how this software handles pedestrians, cyclists, and even motorcyclists. We have no idea how it handles unanticipated situations like black ice or even construction zones. It has not been tested or certified by anyone. Clearly anybody who thinks they can download this, compile it and run it in their car likes playing with fire.

> I support software being regulated just like everything else, not only on life critical situations.

That would be the death of free software. I'm also a P.Eng., but if we're going to require professional licensing, it should only be for safety-critical uses. Shared tools and knowledge have made software systems better and more affordable for countless people. Let's not throw out the baby with the bath water.

Yeah, I doubt that this code dump will stimulate a community to form in a legally safe way similar to the Linux kernel or GnuPGP, the choices of BSD and CC-ShareAlike-NonCommercial are likely to encourage manufacturers to pull a Sony and take the code, build their own training data and contribute nothing back.

Reply on news.ycombinator.com