How would we regulate software engineers? | Hacker News Reader

How would we regulate software engineers?

tavisharmstrong.com

29 pointsRKoutnik54 comments

I have some engineering background and you only have engineers in stable predictable fields. You can predict quite accurately the failure load of a bridge truss or a transistor and its easily testable and the whole process can be simulated and verified top to bottom.

My guess is the wedge to bring in engineering would be network operations, classic operations (not devops which just means we're gonna pencil whip the ops responsibilities). Maybe large data center operations (thermal, electrical loads, UPS, etc) is amenable to engineering workflows.

Software is simply too unpredictable. Both in development and operation. Software is still in the "greeks and amber and rabbit fur experiments" stage of electrical engineering.

The ethical and competency attributes are still relevant. Particularly with respect to actual engineering (architecture of the overall solution) aspects of logic systems development and deployment.

I think one of the main problems with ethics is the abstract nature of software and development. Let's say you create an encryption algorithm. That algorithm can be used by terrorists and pedophiles or it can be used by banks and civil disobedients. Or let's say you create an encryption breaking algorithm. That algorithm can be used by law enforcement and the judicial system or it can be used by identity thieves and oppressive governments. Are you as the developer responsible for all those uses? It is hard enough to know how your employer will use your code in the future, but it would be literally impossible for any open source software.

This kind of argument applies to anything, even paper. Encryption is just a tool. A pipe can be used to hide and carry water, it can also be tampered with to poison someone or conceal explosives.

Which is why we don't force an ethical code on those producing pipes. We instead decided to regulate their finished product. However if someone on HN suggests regulating encryption, they aren't met with the most open arms.

Just a note, looking at the way most companies I've been involved with handle RF testing, it still feels like some electrical engineering is at a premature stage also.

Testing a transistor is the equivalent of testing an if statement. Both are trivial. The difficulty comes from testing multiple of these components acting in sync. Let's be honest, it's sometimes not worth the return on investment to test outside of your requirements.

If software has potential to injure someone, I would hope that it's been tested to similar specifications that hardware is tested on.

You can make software a predictive profession, by adding an order of magnitude of cost to every software development effort. 10x. It's not that it can't be done. It's the market is not suitable for that sort of development. It's a management problem, not a software engineer problem.

There are a huge number of combinations of various statements that have to be iterated over meticulously. If you want to write tests for each and every one of these conditions, it's going to eat a ton of time to do just that and hence add tons of cost to your project. Developers use intuition when dealing with logic, and they make mistakes or don't understand all the possibilities that could occur... because the brain is limited like that. You can't make them not do that. You can just test for it after the fact and fix problems that are identified, that potentially adds huge cost if you are looking to make it fully predictable.

Lots of testing happens in production systems. Especially in open source software. Or during implementation of some library. Those are identified and fixed when identified. To do that within a single effort will add huge cost to the projects.

"You can make software a predictive profession, by adding an order of magnitude of cost to every software development effort. 10x. "

This is a recurting myth. The field of high-assurance software delivers it on a regular basis in regulated markets. They built tools and reusable modules that drove costs down a lot. Still high cost vs throwing crap together.

So, methods were invented far as the 80's to get most of those benefits on the chesp. Cleanroom methodology got very low defects with cost modifier ranging from cheaper to 20 or so percent more extra. That debugging & hard-to-change codd were greatly reduced offset upfront cost. LOCK secure kernel reported about 38% extra cost with full, formal verification & leak prevention. Altran/Praxis does it today for their clients with a mix of Z, Ada, and SPARK for claimed 50% premium. None of these are "10x" higher. Maxed out at 1.5x for most. Market can bear that.

Only proven weakness of highly-assured development is time to market. Features do take longer to build if verified thoroughly. Medium-assurance like Cleanroom has some slowdown but acceptable. Less debugging and breakage by extensions offsets that, too.

  This is a recurting myth.

"Comprehensive formal verification of an OS microkernel" by Gerwin Klein et. al. (2014) [1] reports that it took 20 person-years to perform a comprehensive formal verification of the seL4 microkernel, which was designed, coded, documented and tested in 2.2 person years.

Did they mess it up and take 19 person-years longer than they needed to? I don't know much about this stuff but I thought Klein's work was pretty state-of-the-art? Are there better ways of doing this stuff?

[1] http://www.nicta.com.au/pub?doc=7371

I already told you what it took in numerous projects plus one company's regular work to deliver low-defect software. Vastly lower than that. That means something is missing in either your data ir mine. Thing is, the NICTA papers clearly tell us why:

1. They were mathdmatically proving equivalence of spec, security policy, C code, and assembly. Latter two were state of the art methods requiring more time and brains. Not necessary in most projects to achieve low defects or 0-days.

2. The methods were new. When that happens in formal developmemt, most of the work goes into building tooling and even figuring out how to apply the methods to the class of problem. The seL4 and L4.Verified reports said exactly that predicting that reuse of that tooling would drop cost dramatically. It did drop to a quarter later with COGENT. LOCK, Cleanroom, Praxis, etc reused existing tools and methods where possible to avoid huge costs.

3. Cleanroom, Eiffel, Ada, or even Haskell all cost similar to regular development in commercial use despite knocking out many issues more easily. That they achieve low defects and cost should already establish my claim (strong QA is affordable) as baseline.

> Software is simply too unpredictable. Both in development and operation

You could make it much more predictable if the developer had the legal authority and obligation to sign off on all software. Fuck you I'm not putting my name on that, if you fire me I'll raise a stink about you trying to coerce me into putting my name on something unsafe.

This really isn't needed in most cases though.

Software is unpredictable because there is little demand for predictability. As it is a trade-off, predictability requires experience. Inexperienced application of methods to enhance predictability is expensive.

Speaking from experience:

- Isolate state in modules (micro-services)

- Focus on systems language first, external behaviour second, implementation details last.

- Design systems to fail and fail fast.

- Do not adhere to Postel's law. Be strict in what your systems accept. This forces the systems designer to create a language which is extensible.

And general advice for teams:

- Consistency is the most important factor in success. Choose rarely and choose wisely your: languages / eco-systems / OSes / cloud services / frameworks. Change based on requirements, not HN popularity.

I think its difficult to find this kind of regulation for SE partially because its so new and not well understood compared to other engineering professions. What is software engineering? The kinds of things that would regulate, say, programming an operating system kernel is not the same thing as someone who makes wordpress templates. Yet both would describe themselves broadly as Software Engineers. What is the difference between the aforementioned wordpress-template-maker and someone building a full web application in React? Where is the line that divides "Engineering" begin and end?

I can see a regulatory body develop certifications around software engineers that develop critical infrastructure or spacecraft, where a single failure is catastrophic, but that expertise might not apply for the person making the interface app for the same piece of software (or working in some other layer of the stack that is not so stringent). Is that person "less" of an engineer? It opens the door to institutional bias towards what is considered "proper" engineering, and much like relying on formal education for hiring, provide a poor heuristic in hiring practices.

However I think that we will eventually get there once we figure out collectively what "place" software has in society, once these divisions settle down and become better understood, then we can figure out how to regulate each properly. I think this could be 50-100 years from now.

Wordpress template guy is doing a trade while kernel guy is engineering.

Not all programmers (engineer or otherwise) are guys -- probably worth keeping that in mind if you want this to be a welcoming community for everyone (:

Sorry, i'm canadian buddy. I do call my wife and her girl friends "you guys" from time to time.

I'm Canadian too. I sometimes use "you guys" when speaking to a group that has women in it; I don't use "guy" in the singular when talking about generic technical roles because it implies that being a guy is part of the job. In many cases this is effectively true due to sexism in the industry, so it's not something to take lightly.

I don't think this is what you intended, to be clear! But I do think that is how it would be interpreted, even by Canadian women.

How would we regulate software engineers? Badly.

We don't yet know how to reliably balance desirable goals in software development, such as delivering a certain level of quality and reliability, or delivering on time and on budget. That immediately undermines aspects such as proof of competence or ongoing training that are usually associated with regulated professions.

Moreover, I imagine the people in the industry who would be closest to getting these things right are probably too busy working on building real systems to spend much time teaching or assessing. If government officials started trying to set up a professional accreditation scheme for software developers with real legal weight, how many people here seriously believe the result would not be exams in Agile Software Craftsmanship Manifesto Driven Development, written and assessed by high profile and no doubt highly paid consultants whose total contribution to actual working code in high reliability or otherwise important systems is less than the contributions of at least half the people reading this comment?

We're not ready yet. With luck, we might be within at least some of our lifetimes, but I wouldn't bet much money on it. And in the meantime, any attempt to set this up would surely be subject to instant regulatory capture by exactly the kind of vultures our industry needs to move away from if standards are going to improve.

> how many people here seriously believe the result would not be exams in Agile Software Craftsmanship Manifesto Driven Development

Well, since the sort of people who set up regulatory exams are likely to be the same sort of people who design college curricula, I don't think that would be the case at all. I suspect that a software licensing exam would cover things like algorithmic complexity, NP-completeness, pushdown automata, Turing completeness, LR parsing, etc. etc.

Well, since the sort of people who set up regulatory exams are likely to be the same sort of people who design college curricula, I don't think that would be the case at all.

OK, you win. There actually would be a worse option than letting the consultants do it...

The first thing needs to be a culture change: "move fast and break things" is not an acceptable philosophy when you're dealing with millions of people's private conversations.

OP here. I work at a company that is able to move quickly, and yes, sometimes breaks things in the process. Tradeoffs between speed and safety are fine to make, in my opinion. It does seem like it would be useful to regulate things like ethical conduct, though. Maybe this dictates what systems are ok to move quickly and break and which systems aren't. What do you think?

Probably similar to how other professions are regulated. Educational requirements, internship requirements, licensing, yearly continuing education, etc.

Software engineers are not professionals like doctors, lawyers, or chartered accountants. Real professionals have a degree of autonomy and their primary obligation is to the ethical standards of their profession, not to their employers.

Perhaps that's a problem that professional status would help solve.

> their primary obligation is to the ethical standards of their profession, not to their employers.

OP here. The intended subtext of my post was that it would be nice if our industry had ethical standards as well. (:

That would be nice. It also seems unrealistic in the current climate. What you're suggesting is setting up some kind of Software Engineering Guild, which would require a fundamental shift away from the naive individualism we see today and toward a recognition of collective interests.

You do realize that there is licensing for engineers already? It's on a state by state basis in the US although there is a national organization. It's mostly relevant to the degree that you deal with regulatory bodies and have to sign off on drawings, reports, etc.

In my first job out of school I worked on offshore drilling rig design and retrofits. I took an engineer in training exam and would have eventually gotten a PE (Professional Engineer) license had I continued in that field because you had to submit designs to the ABS and other governmental organizations.

Real professionals have passed laws to restrict supply and prevent competition.

As to the discriminatory and societal problems you write about, I don't know.

But there are certainly standards for particular fields of applications, like medical IT, enforced through stricter liability laws in that field.

In the 1990's and 2000's there was also a very long push for standardization (POSIX/SUSV, programming languages, SQL, SGML/XML, IP protocols, etc.). These efforts (and the success of Java) was also seen as a measure against Microsoft becoming predominant in the 1990's by many developers.

But in this decade, it seems like this isn't anymore a priority. A guy here on HN recently wrote that he'd never consider a language environment lacking a "canonical" or solitary implementation (like, say Ruby, Python, PHP, Perl, or other languages tied to their runtimes have). I found this very interesting, as it's the opposite of what I'm doing (I use only use languages having a language spec and multiple implementations).

I also see it as a generational phenomenon. Look at node.js today. Back in 2011 or so it started as a really practical asynchronous server-side JavaScript runtime based roughly on CommonJS platform specs also implemented by other JS runtimes. Nowadays, the entire reason to use JS in the first place, its ubiquity/portability, is completely lost, and node.js build setups are approaching or even surpassing J2EE-ish levels of complexity/absurdity (webpack, babel, angularjs, etc.).

I think there are a few really big problems. First is that nobody really knows how to build reliable software. Well, maybe somebody somewhere does, but there is no standard established method that can be studied and is known to work. Second, software is a set of instructions to accomplish something. There really is no method to design for a set of unknown instructions. Third, this leads to massive confusion over what design even means with respect to software. Ask 100 software developers what they think design means, and you will get 100 different answers. Fourth, many software projects are reliant on a massive stack of APIs to accomplish anything. It is often not known if these APIs are bug free, or exactly how they are supposed to be used. There are frequently unexpected complications using APIs. Sometimes just using third party APIs becomes a research and development effort. How we build anything is rapidly changing, because the community is still trying to figure out how to do it better. User Interfaces on the web is the most glaringly example. Nothing seems to survive for more than a year or two in this space. Many in the community worked extremely hard to get degrees in Computer Science or something similar and it is not enough. If there was knowledge that worked, wouldn't it be imparted through a college curriculum? In my experience, the upper level managers know nothing about software development and create deadlines that are impossible to meet. Software is something not amenable to being rushed. Code of any quality takes a lot of thought, time, consideration, care, analysis, testing, documenting. It all goes out the window when management starts dictating that the code must ship.

You'd better spend your time, by regulating things that generate code and validating/regulating that output.

Look up DO-178B market. They regulate the software lifecycle itself requiring evidence each part is handled. After source is analyzed and tested, they also do source-to-object code verification. Tools that generate code have to be similarly qualified as you said. An example done by a top company in the domain is below:

http://www.eecs.northwestern.edu/~clk800/rand-test-study/_er...

This question comes up once in a while. One solution to the complete lack of regulation would be to have something like a bar association similar to how lawyers regulate their profession. It wouldn't have to be a legal requirement, but it would at least set a standard for membership.

Free speech. First Amendment covers source code and similar expressions. Regulating free speech is virtually impossible. That is where the whole project stops.

There are regulations that affect the work of software engineers. PCI DSS is one that I am familiar with. Perhaps unfortunately, if your software interacts with the real world (like payments infrastructure), you have to heed regulation. This tends not to affect people who are casually writing software or working on many open source projects, but it does impact large companies like Google.

(I like to think of this as pretty similar to the Haskell IO monad. At some point you have to break out of your cozy side-effect free code and actually do something. At that point you have to deal with the messy real world.)

PCI DSS isn't regulation. It's a standard promulgated by a private LLC (the Payment Card Industry Security Standards Council).

HIPAA would be a much better example of regulation that is material w/ respect to software development.

Great point. Thanks!

> (I like to think of this as pretty similar to the Haskell IO monad. At some point you have to break out of your cozy side-effect free code and actually do something. At that point you have to deal with the messy real world.)

And Haskell is excellent for dealing with the aforementioned messy real world. Better than any of the previous languages (for 1000+ loc) I've worked with.

That's already not the case. Lawyers can get in shit for giving bad advice, and architects have to sign off on blueprints for things that matter.

That doesn't mean that lawyers can't make jokes and architects can't draft blueprints for buildings that smoosh every occupant; it means their professional statements have legal weight.

Good luck with regulating math.

A blanket ban on software developers lying about being engineers would be a good start.

Is there a universally or even widely accepted difference?

The difference between software developer and software engineer is ~$10k. http://www.payscale.com/research/US/Job=Software_Developer/S... http://www.payscale.com/research/US/Job=Software_Engineer/Sa...

lol, you realize that none of the Google "software engineers" are accredited?

Google Canada calls their employees software developers. Everyone would just call themselves a different name and life would move on as nobody cares.

There is a widely accepted difference in the US: the PE exam. Other countries make it flat out illegal to call yourself an engineer without the proper credentials.

I'm not really sure what this would achieve, aside from a shift in the topic of smug comments like this

At this point they are synonymous, so there's no point in trying to change the terminology any more.

That would accomplish nothing but make engineers feel better about themselves.

Gotta ask you, though. How indignant are you about the recording studios using the term "sound engineer?" Or a hollywood post-production studio employing a "special effects engineer"?

I personally would like to retire the phrase "software engineer". This is partly because I think it's very important for software developers to get out ahead of this and avoid giving the various PE accreditation bodies the notion that they have a claim on software. Civil, Mechanical, hell, even Industrial. Go ahead.

But Software developers, if regulated, should stand apart from these fields. I'd rather see it regulated as a separate field, more like actuaries than a branch of engineering.

In short, software developers should drop their claim to engineering, but engineers should drop their claim to software.

Before you think this is a groundless concern, keep in mind that the patent bar has essentially legalized patents on mathematics while excluding mathematics as an acceptable background for reviewing patents. Seriously, the charter for the patent bar specifically mentions mathematics as coursework that does not qualify you to sit for this exam.

The very low quality of patent review reflects this. I could see a PE takeover of software as being similarly destructive (actually, far more destructive).

It does make sense as part of cartel building, though - expand your monopoly, restrict your competition. Regulate mathematics, exclude mathematics degrees. I could see something similar happening in software development, easily, with greater harm.

Regulation is a wrong word. Regulation is something pushed to you from above, something that you did not ask for and that you'll try to evade using that programmer-brain of you.

What we need is something that clearly comes from fellow developers.

Regulation would have to start with simple coding standards. Updated yearly. Much like building codes. A project could pick what standards they wanted to adhere to, and then everyone on the project would have to adhere to those standards -- the whole project would be audited by a third-party after the fact for compliance.

There are plenty of good, short, concise coding standards to draw from. Especially useful in giving non-technical people the confidence that things were built consistently... if not correctly even. This is about as simple as it gets:

* WordPress Coding Standards – Make WordPress Core || https://make.wordpress.org/core/handbook/best-practices/codi...

I'm sure there are others that aren't terrible.

I do think having more standards would help, standards defined by the devs on the project. If they get to choose their benchmark, then it's just about consistency of approach. Not a bad thing, right? Currently it's the wild west, and the only thing a dev can do is say, "It's my word against theirs, their code is bad."

Yeah, this kind of mentality produces utter shit like TOGAF. The code is still crap written by mediocre devs, but now includes at least 100% more complexity and needless abstraction!

TOGAF - The Open Group Architecture Framework [0] for anyone else wondering

[0] https://en.wikipedia.org/wiki/The_Open_Group_Architecture_Fr...

Reply on news.ycombinator.com