Hacker News Reader

It's even worse than that. The standards they run on have contradicting errors in the dozens or hundreds. See this talk https://youtu.be/hXnS_Xjwk2Y?t=899 starting at 15 minutes in (that includes testing for Volvo).

Relevant (for those who don't want to watch video):

- 3000 pages of specs

- 1 million LoC from suppliers (because you use different systems from different suppliers)

- 200 problems

- of which more than 100 problems were in the standard itself (standard being AUTOSAR that described how different parts from different suppliers communicate with each other)

The language won't fix it (it's just a tool after all), though it may enforce certain rules that help the developers not shoot themselves in the leg. Rust has some fairly strong guards, as long as you don't circumvent them. At least it takes some conscious effort to make a Rust program segfault, and almost every IoT appliance (non-automotive, though, I haven't worked with those) I've had that I've reverse engineered had a command that could crash it - I've rebooted my cat feeder way too many times to my liking.

What could be an important change is the culture. At the very least, developers who tend to pick Rust may be (hopefully) more inclined to do things the right way and don't just slap shit on the codebase until it sticks. I've seen way too much of this attitude in embedded developers.

And another important change could be management policies. If they are conscious to pick something uncommon, they might have better tolerance toward spending more time but doing things properly, than pushing people to release whatever seem to work (somehow).

I'm pretty sure a lot of bullshit in IoT happens because management want to release things as fast as it's possible, and developers either don't care or can't fight it.

The language will fix a chunk of it. Much buggy code simply cannot be compiled, and if you can't compile it you can't ship it. You are mostly left with logic and design errors, which can become more obvious due to the language. Middle management will hate it because due to the language they lose much of their ability to compromise on quality, passing the buck. Upper management may love it or hate it, depending on how serious they are about improving quality, as the language forces the spending for that level of quality.

Volvo should actually be contributing to a consortium of automotive computer developers that pool resources together to build shared software on cars. The ultimate outcome would be like an Android-like license that allows car manufacturers the freedom to build-on-top of a software stack that gets security updates.

Cars get tricky, though, because bugs can lead to loss of life.

I bet Android Auto devices get even less updates than most Android based phones and tablets.

Isn't Android Auto just proprietary audio/video/touch protocol over USB-C, with no actual Android bits on the car side?

This is the case for a lot of engineering, I've seen it time and again.

The business processes force the engineering work product to be a dumpster fire.

The great companies dont.

> Concurrency problems often are not debugged; locks and similar are just sprinkled around until they disappear in testing.

It does sound like Rust could help here

Not with liberal use of `unsafe` it can't.

IME It's often harder to do it the unsafe way rather than to just do it the safe way

At the end of the day it's embedded code that's inherently unsafe, so the benefits are all in the code that belongs in the safe zone. Unsafe Rust is at best equivalent to C/C++ and at worst it's a false sense of security. Mixing safe and unsafe without a clear boundary is not something I can speak to with confidence but my gut-feeling is that it's worse than C/C++.

Even OS kernels are borderline, and it's still being figured out how exactly Rust fits, but yes we are slowly moving in that direction. Kernels have a lot of logic that qualify for the safe zone, and software engineering culture that is... let's say ahead of automotive.

> embedded code that's inherently unsafe

I'm not really an expert in embedded systems, so take this with a grain of salt, but that doesn't track with my experience or with my theoretical understanding of embedded systems. First off, embedded systems are just regular systems with tighter constraints and less high-level constructs. With that understanding, if embedded code is unsafe... so is any code on any system.

Second, embedded systems have peripherals that are accessed via direct memory reads and writes. Which is only unsafe if access to those peripherals is shared in an unsafe way. Sure, the rust code that reads and writes to memory addresses will use `unsafe`, but everything above that can be written without using unsafe. Which I guess makes your statement technically correct, but imo not practically correct

> Unsafe Rust is at best equivalent to C/C++ and at worst it's a false sense of security.

This is a bit of a misconception for people who've not worked with unsafe Rust. It's not like the language and most of its features disappear. It just allows a few extra features that are unsafe, like working with pointers... Which, granted, is a lot of interaction with hardware. But you can kinda "cordon" this to a small subset of the codebase dedicated to this low level hardware IO

I would not trust your gut feeling.

If anything Rust has a clear boundary, and that's useful for all kinds of static analysis.

And I say this as a fan of D lang. I don't really see any advantage of Rust over D except popularity.

It's not like people actually do this though. I've been programming Rust for 5 years now. Since three years 'mostly Rust', now 'only Rust'. As far as I've seen `unsafe` is used very sparingly in every codebase I've had to deal with. Mostly when interacting with C, as there's no way to call C without `unsafe`.

But even then that's usually abstracted away and then a safe interface being provided.

It depends if the same pressures that lead to sprinkles of locks will lead to sprinkles of unsafe.

Maybe, but the thing about unsafe is, it isn't like fairy dust so sprinkling it on stuff doesn't help. This is a common misconception from people who haven't worked with unsafe (even if they've written some Rust)

ten_things[11] = blah; // is a buffer overflow so it either won't compile or it panics

However

unsafe { ten_things[11] = blah; } // is still a buffer overflow, same effect but also Rust warns that unsafe is useless here and should be removed

Now of course we can write a potential buffer overflow in unsafe Rust, but now we're not just sprinkling unsafe on stuff and hoping, we've got a plan, perhaps a stupid plan but it's a plan so that's an improvement.

I agree with you but just for completeness the rust equivalent is unsafe { *ten_things.get_unchecked_mut(11) = blah }

Which goes to show that usually the unsafe methods are more verbose and annoying to deal with. Devs won't reach for them just out of laziness.

I'm sure you're much more knowledgeable than me about Rust at least.

Still based on my experience reading early Redox code it was chock full of unsafe blocks, and I suspect they didn't do that out of malice but to expedite development.

So the risk might still be there.

Redox: “ A quick grep gives us some stats: the kernel has about 300 invocations of unsafe in about 16,000 lines of code overall.”

https://doc.redox-os.org/book/ch01-07-why-rust.html

This is probably not an up to date number but it gives you some idea of a random snapshot in time at least.

Well if I knew less than the last guy I'm defo beat here :)

Would be interesting to see what those figures were like 5 years ago (i.e. a year or so after its initial release). It seems like the concern here is that embedded Dev for cars isn't getting getting to maturity.

I don't see why it would. Extra locks help paper over bugs you can't figure out. Unsafe would never do that.

I have no experience in automotive software development so I could be very wrong about this. But I suspect that Rust is a very poor choice if what he said is true about automotive development since, at least some, Rust devs openly talk about Rust being a more complicated language to program in. Increasing the complexity of software development in an environment like that will ultimately add the problem instead of solving it.

But referencing the parent comment's discussion of business processes, its highly unlikely the problems will appear until Volvo tries to move to some other vendor or swap out the software solution for the next greatest thing.

> Rust devs openly talk about Rust being a more complicated language to program in

More complicated than what, C++ which is probably what was used before? It's unlikely to be more complicated in that case.

Only for concurrency problems inside the same process space, for concurrent access to external resources little help is provided.

How many automotive code bases have you seen and in what sort of capacity? Also, how recent?

Generally speaking they also don't pay software development well so I would guess they aren't exactly getting the best talent either. I think cars are cool, but I wouldn't take a 50% pay cut to develop software for them.

It's not a question of talent, just timing and process. The timing is rigid and there's a hard, non-negotiable deadline when metal starts being stamped. The process around certifying makes you work at a tiny fraction of your normal speed, because most time is spent filling out auditing requirements.

You still need to pay enough to attract the kind of people who understand what "undefined behavior" is and why you should avoid it. That number is somewhat higher than what most mid-level automotive programmers are paid, in my experience.

Cars have been hardware+software for some time now, aren't they? ECM have software and have been around since at least 1998.

> Choice of programming language is the least of their concerns, that isn't the source of their code quality issues.

I'm curious to see if the Rust programmers affect the organizations as well by introducing unions to combat this kind if thing.

On another note I disagree that language will have no meanigful effect, even in a hostile environment such as this.

A substantial amount of this development is already done in managed/safe languages when possible. In many cases, the software developers are represented by a works council or similar. These are generally capable and skilled developers that want to do a good job, not half-literate code monkeys slinging bad C in a sweatshop.

Whatever ethos you imagine Rust developers uniquely have, it doesn't address any of the constraints and incentives that are creating the code quality problems in automotive. Even the best developers in the world would have a difficult time producing a quality product in that environment.

> Whatever ethos you imagine Rust developers uniquely have

They tend to seem more of the type to go on strike, form a union, or leave a job because of political views.

> Even the best developers in the world would have a difficult time producing a quality product in that environment.

Right, i'm saying I think at least more experienced rust developers would refuse to work in that environment.

They seem like they would risk more of their own well-being for ethical reasons, for better or worse.

I suppose if Volvo started depending on highly socially conscious domain experts and had a choice between changing working conditions and ditching rust though, they'd change the latter.

I'm happy to be proven wrong, but experienced C programmers seem to be more conservative "don't mix politics and programming" types.

> I suppose if Volvo started depending on highly socially conscious domain experts

It's amazing that you're saying all that and the same time completely oblivious to the fact that Volvo is in Sweden. Which has unions, labor protection, amazing working conditions etc. etc.

The reason Volvo is doing what it's doing is due to Volvo's commitment to safety [1] and not because of some mythical exclusively social ethos of some programmers.

[1] https://www.volvogroup.com/en/about-us/traffic-safety/safety...

Maybe in US, in Europe unions tend to be a common thing.

German car manufacturers already have unions. Unions don't help against overmanagment and leadership without technical (digital) understanding. "Creative" parts of the company are often praised (while mostly badly stealing ideas from the competition) while devs are seen as a "resource". Experts usually get asked AFTER the project is on fire or already crisp.

I think you are on something. Tangentially I used to sit next to a colleague who was responsible for research software development in Python. The amount of necessary coordination in developing such software made the dev mental health to suffer. In contrast in Julia which I was already using at the time development is more autonomous as things just work together.

In Germany we already have unions, including the software development, as the unions are traversal to the industry regardless of the role each employee has on the building.