Hacker News Reader

Comment by dmreedy | Hacker News Reader

I believe these are the kinds of ideas that might actually create a genuine engineering culture in software development. Until we start applying this kind of rigor to our work, I don't believe the title of "Software Engineer" is justified. It doesn't have to be TLA+; it doesn't have to be any particular tool or technology or pattern or whatever. But the attitude that rigor and formal technique is worth the additional effort is something that I don't find often in the current culture, or at least the narrow slice of it that I've experienced.

The quote at the end sums it up nicely. Coding should be the easy part. A sentiment everyone tends to nod their head along with, but rarely has the discipline and willpower to implement.

View on HN

I think that tools such as TLA+ show their value not by convincing people that they're worth the extra effort but that they actually save you effort. The experience at Amazon and Microsoft shows exactly that, and managers were relatively easily convinced.

Absolutely agree. I work in the autonomous vehicle space, specifically in creating high-assurance resilient systems, and I've had more arguments that I care to count which went something like the following:

Them, "All that extra effort sounds great, but we don't have the time to do that. It will explode the really tight build, test, debug cycle we have now. Suddenly every cycle will be 100x as long."

Me, "First of all, you should be less concerned about the length of an individual cycle and more concerned about the sum of all the cycles up to completion."

Them, "Okay, but having short cycles also helps us course correct quickly, and keeps our 'velocity' up."

Me, "Well, second of all, what good is it to be able to iterate really quickly toward an unspecified goal? How do you know that what you're doing will get you there? Or worse, how do you know that what you're trying to do is even possible or not? Maybe you're 'engineering' your way asymptotically toward an impossibilty result."

It's sometimes crazy making to have these conversations and see how wildly distorted we've allowed our incentives and perceptions become as a group of professionals. It's become more important to produce the appearance of progress and effort than it is to actually make progress and thoughtfully apply effort.

Either your frustration is great, or you have the mind of a jedi. This is without a doubt the most scathing review I've read of agile, and you don't seem to even be trying.

The problem is, most of the people who you are working with, only have 0-2 years of experience in the technology that they are using (use of research / deep learning to build products). Look at their resumes. They'd be either recent PhD grads, who haven't built a single product in their life. Or traditional engineers who had just joined the field of building products with an approach that is very different from one that they are experienced in.

I would also caution you. Do you really know the technology that you are trying to make safe? Building systems with research / deep learning is as different from traditional software development as synthetic biology from electrical engineering.

That's an institutional problem though.

Yes, it's true that most companies hire way more people, way too quickly than is probably prudent for a problem domain (especially one that is still such an open-ended science project like AVs), and that as a result barely anybody has any idea what they're doing measured against what's being expected of them, but that's a wound that is entirely self-inflicted. It's not a natural law that one needs to suddenly staff up dozens or hundreds of really expensive people ahead of any real capacity to train or assimilate them, but it does happen with depressing regularity. If anything, I'd say this is probably the biggest problem I see in the ecosystem that you and I are in.

The team I work with is small. Everyone has several (5-15) years of experience consuming research, turning it into practical solutions, and in some cases even having created a novel practical application that was later supported by adjacent research.

It's not easy to construct a team of half-dozen to a dozen prolific programmers w/ a bent toward the academic, but an innate sensibility for remaining pragmatic and applicable. But, it's also not impossible to do so either. This is inline with my incentives comments elsewhere. I think the incentives are broken at too many layers to be able to be fastidious and intentional about how we solve problems. Say you get an "empire builder" into the leadership structure... you're doomed. The whole point of everything becomes about having as big of a "rockstar" team as possible, not about trying to solve a problem. Unless the problem one is trying to solve is how to quickly become the SVP of an extremely expensive and probably fairly unproductive team of increasingly disaffected previously very effective people.

I'm finding a fifteen years figure hard to believe. There was a handful of people at that time that have invented deep learning. It took years to get any industry use, as the technology was not yet competitive, until AlexNet.

We don't work in AI/ML, we work on high-assurance distributed systems.

That said, you'd be right. Not 15-years in something like computer vision. No such thing as formally verified NNs, regardless of what nuTonomy claims, exist. Even the basis about what one would be trying to verify is an open question. It's partially the point of DARPA's Assured Autonomy program.

The problem at the moment is that the ecosystem has two science projects on its hands. How to make an ML-derived driver is one. How to make a stable, correct substrate for acquiring data, make a decision, and react correctly is another. The only one most people hire up for is the former, and the latter is at least as challenging.

Isn't it a "high-assurance distributed AI/ML system"? It is interesting that there is indeed a tendency to factorize it. I believe, this tendency arises because there are two groups of people that are available - traditional engineers, that are excellent and experienced, but do not know the language of AI and can't tell an ensemble from an embedding. And AI engineers, who are yet to build their first product.

Sort of. Except I don't think there's a such thing as "high-assurance AI/ML". Put simply for the audience here. The goal is to have the whole stack from top to bottom be as predictable and deterministic as possible.

There's a layer of it (the AI/ML bits) which is necessarily probabilistic. That's by design. It's how it all works. All the layers below that today are currently terrifyingly probabilistic. It's difficult to advance the capability of the whole domain when the necessarily probabilistic parts are depending on shifting sands as a stable foundation.

I agree though that there's a large gap there in the domain expertise of the parties involved, and that's produced significant challenges as we've not yet figured out how to close that gap.

So, two questions:

1. What parts of AI/ML have to be nondeterministic?

Once the model is trained then it should be completely deterministic. There are some training techniques that randomize the order or grouping of data, but they are primarily used because random is almost as good as finding the optimal ordering/grouping and finding the optimal ordering/grouping is hard. Sometimes randomness is used to augment the data to give you more examples to for against (because the model is too complex to train/fit with the data you have it you want to be robust to certain variations in input. If you're talking outputs then you should get the same outputs from a model given the same inputs. There are some really bad models out there that are way over fitted (relevant xkcd - https://m.xkcd.com/2048/ see "house of cards") and some people working on ML have a terrible grasp of statical techniques (just try models until they find one that fits), but even these models are deterministic.

What is so bad about nondeterminism anyway? Ethernet used nondeterminism to handle collision resolution. There best prime finding algorithms are nondeterministic. There is nothing wrong with appropriate nondeterminism.

Are you sure, such thing as "high-assurance AI/ML" doesn't exist? Are you an expert in AI, with many years of experience building products with AI technology?

I'm not an expert in AI, but as you can imagine I have to interact with a lot of people who are. What I do know is what it means to be able to do end-to-end verification and it seems fairly antithetical to the basis of machine learning which has an open-ended failure domain. If you could write an AI as a total program, it would cease to be probabilistic. It would become propositional, but would also seem to cease to be writable and computationally tractable. As it would need to encode all possible states of all possible features of all relevant domains and all possible things that can interact with each of those things and what their effects would be.

Though, I'm happy to be proven wrong here as it would make me a lot less nervous about the future were there a way to legitimately assure the artificial driver in an AV won't fail in emergent ways, or at all.

I work on the formal verification of AI systems as well, but focusing on model-based reasoning systems rather than the model-free ML systems. What you've described is exactly what terrifies me most about ML in safety critical contexts: it can fail in a large number of ways, and predicting these ahead of time is generally intractable.

I like to think of it in terms of quantifiers: for most systems it's fine to say that there exists a set of representative tests that the system passes. For critical systems, you really want "for all" in there instead.

Fortunately the model-based systems are much easier to reason about, even if they suffer from difficulty modeling certain cases. For a model-free system I think the starting point would be to precisely state a correctness criterion that could be used as a decision procedure on the output of the model; the combined system would be "verified" to a certain degree. Unfortunately you can't guarantee that it would always be able to yield an answer (it might always give a bad one which was rejected) and, the more fundamental problem, one might reasonably expect that if you can express the correctness property to a sufficient level of detail that you can perform verification on it you might be able to write an easier-to-verify implementation that doesn't rely on ML.

This stuff is hard, yo.

"What foolishness! What nonsense! .... For as children tremble and fear everything in the blind darkness, so we in the light sometimes fear what is no more to be feared than the things children in the dark hold in terror and imagine will come true. This terror therefore and darkness of mind must be dispelled not by the rays of the sun and glittering shafts of day, but by the aspect and law of nature."

LUCRETIUS, De Rerum Natura

I think your frustration largely arises because the problem you work on is uncommon. The goals you are working towards are very large and error is not an option because people die or at least massive property damage occurs. Most applications aren't like that. It doesn't matter much if 1/10k image upload to Instagram fail. We have a lot more Instagrams than autonomous cars so the engineering culture conversion is largely dominated by those trade offs. It's interesting because I think we see experiencing the pendulum having swung heavily towards Agile. In the past we used to slow down some crud apps by first drawing UML diagrams. Fingers crossed that we can discover a wider, more differentiated range of engineering practices.

My particular vein of work certainly makes me more likely to have interacted with this different way of doing things, but I don't think at all that it's a necessary component to there being reasons to consider it and reap the benefits of it. Because I didn't always used to be in this vein of work, and in retrospect I would have benefited enormously from being more judicious and fastidious in my prior lives using things like TLA+ or requiring even super basic things from my co-workers/teammates like property-based tests. I would have saved so much time and money and heartache over the years.

It can't possibly be more efficient to take 50 fairly expensive people and have them iterate 10,000 times toward something that still ends up being fairly broken---and requires maintenance in perpetuity---than it is to take 6 slightly more expensive people, and have them iterate 10 times toward something that does exactly and only what it is supposed to do and requires basically no attention once it's been instantiated, can it? Even if you're Instagram.

> It's become more important to produce the appearance of progress and effort than it is to actually make progress and thoughtfully apply effort.

That's why I'm sceptical about going to work on the autonomous vehicles tech (although it is tempting). I'm affraid there's been so much money and hype around it that "failure is not an option" - i.e. companies will sooner see another Challenger disaster than admit that they're not on track for their announced goals.

Based on your handle and your concerns, you should email me. :-)

I absolutely would, but how do I reach you? :) There's no contact info in your profile.

spam.is.food.not.mail@gmail.com is a reasonable way to reach me.

^^^ I'm quite serious.

>the attitude that rigor and formal technique is worth the additional effort

Commercially speaking - it's usually not.

Does this speak more to the pursuit of the application of software to problems generally or to the baseline triviality of problems that are funded to have software applied to them?

To me it seems like an incentive problem that starts with the distorting effects of unsophisticated investors w/ unrealisitic expectations who fund following a herd mentality only things they can themselves pattern-match to.

It speaks to economics.

A business process that works "by itself" most of the time, but requires human intervention once in a while, is still a functional business process. The unit of "software + 5% of this person's time" is perfectly workable. And if the cost of the fix is more than the NPV of 5% of that person's time, it isn't worth making.

Catastrophic failure risk, of course, changes the analysis. But a huge range of things are not catastrophic in their failure modes.

To me it speaks of Software being "good enough" far more easily than it can be perfect.

> To me it seems like an incentive problem

Why is it a problem if a buyer gets something that works well enough for a lot less time/money put in?

Further to that - specifications and requirements are often imperfect, poorly pinned down, ever-changing. Software creation out in the commercial world has shifted working patterns to try to adapt to that (Agile methods amongst others). Attempting formal proof of such soft, malleable systems would appear to be a fool's errand.

I'm not sure I follow.

The challenge with this formulation of "ever-changing" is that it assumes that things like Agile methods are somehow creating the capacity to iterate toward something of value with intention and ambition in such a way that you'll achieve whatever it is that you're trying to achieve.

However, what happens is that the lack of rigor in software development today seems to cause the lineage of development and associated created technology to drift further and further away from the desired commercial goal as those goals change or mutate.

Creating a bunch of brittle technology and technical debt to poorly approximate Purpose-A (a purpose by the way that you might be able to formally prove trivially is actually impossible and you either shouldn't waste the money or should intentionally select the best sub-optimal solution) doesn't help you get to Purpose-B any faster. As you become aware that Purpose-B is a thing you need to angle toward, you're still dealing with the pile of bubblegum, toothpicks, and instruction manuals written in alien script that still hasn't achieved Purpose-A.

It's a really weird thing to me that as an entire ecosystem and community we've decided that the most valuable way to approach software is to declare that it's impossible to ever "finish" it.

If I have a series of high-assurance layers to build upon that I know I can trust, then iterating and adapting to changes actually becomes faster and easier, because I know the surface area of where a problem can actually reside and how deep in any part of a system I actually have to uproot and start over. In the current formulation of the world, basically everything is a suspect, and basically any solution is considered viable... whether it truly is or not.

I've been in situations where Product Managers and Engineers spent months off and on arguing over some proposed solution or feature and never getting anywhere because everybody's basis was built on social capital, political capital, and mythology. What finally killed the conversation and allowed progress to actually happen? When a TLA+ model that invalidated one of the proposed hypotheses was presented. "Well, look. The way you want to do this is actually impossible. It will need to be done like this, this, or this instead." Seven... _seven_... sprints of about a dozen people wasted because nobody could formalize either the requirement or the approach. Until somebody did. This wasn't for a safety-critical system either.

> The challenge with this formulation of "ever-changing" is that it assumes that things like Agile methods are somehow creating the capacity to iterate toward something of value

If you're doing it reasonably well, they produce small chunks of something valuable every iteration.

> Creating a bunch of brittle technology and technical debt

Who's creating brittle software or technical debt? Things can be well designed and well written, well structured and well tested without going as far as formal proof, which in most commercial situations is going to be masses of overhead and quickly not applicable to the ever-shifting problem domain.

> It's a really weird thing to me that as an entire ecosystem and community we've decided that the most valuable way to approach software is to declare that it's impossible to ever "finish" it.

You seriously need to get out into the commercial world more. Or perhaps into the smaller, faster moving commercial world (I'm not sure where you are now).

> If I have a series of high-assurance layers to build upon that I know I can trust,

Then the underlying assumptions are likely to change, rendering your layers an expensive boondoggle.

> I've been in situations where Product Managers and Engineers spent months off and on arguing over some proposed solution

Then you have bigger problems than formal proof.

I'm generally looking at things which are designed and delivered within a number of months - anything that takes months to argue over has entirely missed the boat as far as I'm concerned.

It will be, we just need a couple more lawsuits about faulty software and security exploits.

How long has this sort of stuff been said though?

Not long enough, but IoT mass deployments are going to change that, as they are much worse than Windows 9X security issues.

I'm not sure that we'll end up with more formal proofs as a result of this. But better locked down systems would be a good thing in this space.

Totes agree. Also, customers are humans. They don't want to be handed a perfect product. They want to be involved in the process. Make dumb decisions. Complain. Receive bugfixes. Repeat.

I don't know. Maybe I'm jaded, but I think in B2B, good software is not even desired. In B2C, sure, as Apple proves, but not in B2B.

> good software is not even desired.

You can make good software without formal proofs. It just may not be proven to be perfect.

> They want to be involved in the process. Make dumb decisions.

This I agree with though, sure. They do make dumb decisions, and they push ahead anyway, even when told. Later they change their minds, we change the software. It's the grand circle of life or something.

I disagree that engineering is the best approach to writing software. I think this is the fundamental difference between people who prefer static vs dynamic typing. People who prefer static typing are more likely to say that one solution is more "correct" than the other, whereas dynamic typing allows more flexibility.

I see writing software to be similar to writing music. Music has rules and structure, but leaves room for creativity and allows two or more highly skilled musicians to be able to play music together and improvise.

Static typing imposes limitations (kind of like government), whereas dynamic typing gives you more freedom (but requires responsibility)

It's fine when we're talking about leisure-paced art, done by people who deeply care about what they do. But the reality of software development is different. Software is written primarily by people who treat it as a job, and care about it (on average) about as much as they're paid to - and they're being paid by people who don't care about the product, but its ability to make money. Software written in such environment is (unlike music) powering systems that affects livelihoods, health and safety of other people. With the market pushing quality down, we need a countering force pushing it up.

Or, imagine dropping all regulations around planes, cars, food, health care or infrastructure. Would things be handled better by artists, free to pursue quality? Maybe. Will majority of the providers be artists, or even care about quality? No.

Just a sidenote, reading your comment it occurs to me that I have no idea - and as far as I can tell never read anything about - what is the proportion of code that is done 'for fun' versus 'for pay'.

If anyone has any survey or any discussion on that please reply so I can take a look at it. I'd like to confront my expectations with data

You're conflating the act of building a product with the act of writing code. Even in the most expressive of languages, there are objectively good and bad ways of instructing your hardware to perform a particular calculation or set of calculations - and in the more rigid languages there is often a flat-out right and wrong way.

Design your product with as much free thought and creativity as you please, but never let yourself believe your actual code is art. That - among many other outcomes - is how your platform gets compromised and your customers' identities stolen - not to mention how the successors to your job end up cursing your name and wasting time and money replacing your code wholesale.

> I see writing software to be similar to writing music. Music has rules and structure, but leaves room for creativity and allows two or more highly skilled musicians to be able to play music together and improvise.

Bad music carries only the risk of people not liking it. Bad code on the other hand...

Usually risks people not using it or playing your game.

Rarely does it kill and in those cases (self driving cars, banking) those application usually has a company behind them.

If the RSS reader you're building has no guarantees of working it has no value to anyone. Not all software needs to try and be bug free but all software should strive to minimize the number of bugs in it.

Static typing helps to minimize bugs (when used correctly) by allowing function contract declarations and reducing the build up of defensive guards over time. I've seen code in dynamically typed languages with lines like `if (!foo is integer) raise;` which basically equate to onerous home-built static typing.

If it has no guarantee and it works it still has value for whoever uses it. If it didn't work it might have less value but still has some value.

Why would guarantee free software have no value. It's like buying a used phone.. no guarantee but it still has value.

Parent isn't saying his way is best. He's saying it's the appropriate mindset for some types of software. What you describe might make sense for bespoke web apps (and many other applications), but it may not be appropriate for, say, firmware development, or writing a web server from scratch.

That might work fine if you're throwing together a blogging system, but not in a critical-systems software-engineering context. When the stakes are high, it doesn't do to view your code as a creative outlet.

Critical-systems software engineering is not about your creativity, it's about building maximally correct software, to minimise the probability of a catastrophic outcome.

This idea that it's inherently virtuous to let software developers be creative, strikes me as quite wrong-headed. If a strict methodology produces better results (and it does), a strict methodology should be adopted.

Bridge engineers don't whine about their constrained creativity. They're too busy ensuring their creations don't get anyone killed.

With the curious exception of Erlang et al, few people serious about program correctness favour dynamically typed languages. There's a reason why Ada is statically typed, and strongly typed (it lacks the dangerous implicit type conversions permitted by C).

An excellent article on this kind of thing: https://www.fastcompany.com/28121/they-write-right-stuff

> Static typing imposes limitations (kind of like government),

where to begin

I program as you say 'similar to writing music'.

This is because given all of the levels of various languages, frameworks, and libraries- I'm just trying to get things to work.

However, I would very much like the methodical engineering approach at my day job applied to programming.

It really blows my mind we don't have full stack + auth that works cross platform out-of-the-box. It blows my mind my recent framework didnt show the post request exchange to get tokens and authenticate.

There are NOT that many people that program at this level. It seems most of my programming questions are answered by asking experts rather than reading a wikipedia article. Electrical and Mechanical Engineering are better solved reading rather than asking engineers. IMO.

But it's clear that one approach can be more correct than another. If it was supposed to save a document and instead it deletes all my documents, it's not right.

I hope you stick to webdevelopment and never touch anything remotely related to medical devices, or control systems.

So you're saying Python devs are all libertarians? :D

I agree that coding/development is an art, but I also am hard pressed to find anything that, when done well, isn't an art.

The difference is that music doesn't have any really notion of correctness or efficiency; sure, there are ideas such as "no parallel fifths" and "tritones must resolve to M3 or m6", but this isn't an issue of correctness, but rather stylistic guidelines. Schoenberg's compositions weren't incorrect in any universal sense.

Programs, on the other hand, can be incorrect or inefficient. This is because most programs have a concrete function that they produce.

Dynamic types are great for exploring your solution space. I can quickly whip together a prototype, try out different solutions, etc. I don't need to design my entire program ahead of time, and this is nice. However this does not prevent me from reasoning formally about my program. Once I've muddled around a bit and built up some scaffolding, it very well may behoove me to try to shore up my design, spend some time reasoning and designing a formal algorithm/specification/etc, etc.

And for the record, as a musician I definitely think formally about my compositions/pieces I'm playing. Granted, being a guitarist this usually doesn't involve me writing out my pieces with sheet music, but there is still a blend of intuition/exploration and brute computation.

As an example, I often times have two disparate parts that I need to connect together, maybe by modulating, maybe by changing registers, or maybe by altering my time signature. At that point I put the guitar aside for a moment and think about my formal requirements, and then try to find some 'convincing' way to get from point A to point B. Maybe I have some constraints (I have two measures, I need to start in this position and end in that position, this finger needs to pedal this note, etc). This quickly becomes a math problem where only a small set of viable solutions exist. After 'doing the math' I am left to evaluate my solutions aesthetic quality which, I suppose, is where most people think of the 'artistic' bit comes into play. But really, I'd say artistry abounds throughout the process.

And to be clear, I'm not thinking about this stuff as doing math as I'm doing it--I have my own system for thinking about this stuff that doesn't look like math.

Anyway, all of this is to say that formalism can be used even in music with very real benefits.

</brain-dump>

Isn't that why you have titles such as Software Engineer, Developer, Programmer? Software Engineer should have the engineering background and tackles hard problems whereas Developers and Programmers focus on products and features.

All of these terms are used mostly interchangeably, and it just depends on what company you are at. For example, Microsoft calls them all SDEs (Software Development Engineer) and Google calls them Software Engineers.

In many countries it is illegal to do so unless those people are actual engineers with accredited degree by the engineers' college.

Reply on news.ycombinator.com