You are never taught how to build quality software
florianbellmann.com
florianbellmann.com
I get that this is a blog post and needfully short, but yes, there are courses that teach these skills. There's a big disconnect between CS and SE in general, but it's not as bad as "no one teaches how to build quality software". We do work on this.
The biggest impact I've had at the places I've worked have been about procedures and methodology, not how to use UML or draw a dataflow diagram.
- Have a process around software releases. Doesn't matter what as much as it has to be repeatable.
- Review your designs, review your code, don't do things in isolation.
- Have a known location for documents and project information.
- Be consistent, don't do every project completely differently.
- Get data before you try to fix the problem.
- Learn from your mistakes and adjust your processes to that learning.
- And many more things that sound like common sense (and they are) but you'd be amazed at how even in 2023 many companies are developing software in complete chaos, with no discernible process.
This was a thread about why software developers don't do engineering like other disciplines. One partial answer is that those other disciplines take it much more seriously at the undergraduate level, at least on average.
Probably the more compelling answer is that the industry doesn't' really want them to for the most part.
> but you'd be amazed at how even in 2023
I really wouldn't.
This is it. Everyone is making money hand over fist despite not doing it. You might want it, but you don't need it.
I haven’t touched UML for ten years.
I'm no fan of UML, but the exercise is to teach students how to plan, how to express that plan, and how to reason about other people's plans. The students will certainly draw a lot of flow diagrams in their careers, and will almost certainly deal with fussy micromanagers who demand their flow diagrams adhere to some arbitrary schema that has only limited impact on the actual quality of their work or documentation.
UML is complete, at least.
It's not clear to me what your argument is. Is it using whiteboards to draw uml instead of special uml software? If so, be prepared to take much longer to draw the diagram.
Or do you mean uml is deficient compared to free drawing of symbols on a whiteboard without a standard? If so, be prepared that nobody will completely understand your diagram without explanation
No need for a specific tool - unless you’re doing PowerPoint slides. Visio is good enough in that case if you have Windows.
All I'm arguing for here is that UML serves the same purpose as those online homework apps. Correctly formatting your calculus homework to be accepted by that interface is as unrelated to calculus as UML mastery is to effective software design, but it resolves some of the same logistical challenges, while introducing others.
I mean, yeah. Seeing that wave happen over the course of their college career would probably be better prep for a career than most CS classes.
I did not read the book "a philosophy of software design", but I just scanned the table of contents, and it is not clear to me how "a philosophy of software design" contradicts with using UML.
Are you telling me that in those 7 years of working, you never once used a class diagram? Or a database diagram? Or an activity diagram, deployment diagram, sequence diagram, state diagram?
I find that hard to believe... how do you explain your design to other people in the team? Or do you mean that you do make that kind of diagrams, but just use your own conventions instead of following the UML standard?
Personally, I often use UML diagrams when I am working on a technical design and I use those diagrams to discuss the design with the team. On almost every project I create a class diagram for all entities in the system. I rarely make a database diagram because that often is a direct translation of the class diagram. For some of the more complex flows in the system, I create an activity diagram. For stuff that goes through state transitions, I create state diagrams. In my case, this really helps me to reason about the design and improve the design. And all those diagrams are also very useful to explain the whole software system to people who are new on the project.
That does not mean that my diagrams are always strictly UML-compliant or that I make a diagram for every little piece in the software system. The goal is not to have UML-compliant diagrams, but to efficiently communicate the technical design to other people, and UML is nice for this because it is a standard.
I agree that all those diagrams are also very useful to explain the whole software system to people who are new on the project, however, that doesn't imply that having this ability is common, many (IMHO most, but I don't have the data) companies intentionally don't put in the time and effort to maintain such up to date diagrams for their systems.
I was a lot faster than my roommate, and after I turned in my homework I’d help him debug (not solve) his. Then I was helping other people. They really did not get debugging. Definitely felt like a missing class. But it helped me out with mentoring later on. When giving people the answer can get you expelled, you have to get pretty good at asking leading questions.
Then I got a real job, and within a semester I was down below 2 hours. We just needed more practice, and lots of it.
Which means you learn more in one month of being on a normal, 40 hour workweek job than you have in an entire semester of one course.
Huh? I was spending 20+ hours a week on assignments alone in upper level software engineering classes.
Also, internships were required.
My group did the latter. I think based on what we know now about interruptions, we were likely getting more done per minute than the responsible kids.
Including reading, we might have been doing 15 hours a week sustained, across 2-3 core classes.
But these were the sort of people who got their homework done so they could go back to the ACM office to work on their computer game, or work out how to squeeze a program we all wanted to use into our meager disk space quota.
Anything more than a B was chasing academia over practical knowledge. B- to C+ was optimal.
If you observe a car mechanic trying to find the problem with a car, he would go like: "is this pin faulty? No. Is the combustion engine faulty? No. Are the pedals faulty? Yes." where the mechanic starts with some assumptions and disproves them by testing each of those assumptions until (hopefully) the mechanic finds the cause of the fault and is able to fix it. Similar types of investigations are important to how natural science is done.
So it would be helpful if you can clarify your intended meaning a bit more. Maybe I or someone else would learn from it.
And I don't feel is not something that is worth teaching in universities, because it is 90% experience and for me, the point of universities is not to replace experience, just give enough to students so that they are not completely clueless for their first job, the rest will come naturally.
What universities can teach you are the tools that you can use for debugging: debuggers, logging, static and dynamic analyzers, etc..., different classes of bugs: memory errors, the heap, the stack, injection, race conditions, etc..., and testing: branch and line coverage, mocking, fuzzing, etc... How to apply the techniques is down to experience.
In fact, that's what I find most disappointing is not juniors programmers struggling with debugging, this is normal, you need experience to debug efficiently and juniors don't have enough yet. The problem is when seniors are missing entire classes of tools and techniques, as in, they don't even know they exist.
Good points on the tool side. While "debugger driven development" is rightfully considered an anti-pattern, the tool-shaming that sometimes emerges from that consideration is a huge mistake.
It is true that initially you may not know where the bug is but you have to collect more evidence if possible, see if you can always cause the bug to manifest itself by some tests, explore it further, the goal being to form a hypothesis as to what the cause may be. Then you test the hypothesis, and if the test fails then you form another hypothesis. If the test succeeds, you refine the hypothesis until you find what is going wrong.
Such hypotheses are not formed randomly. You learn more about what may be the problem by varying external conditions or reading the code or single stepping or setting breakpoints and examining program state, by adding printfs etc. You can also use any help the compiler gives you, or use techniques like binary search through commits to narrow down the amount of code you have to explore. The goal is to form a mental model of the program fragment around where the code might be so that you can reason about how things are going wrong.
Another thing to note is you make the smallest possible change to test a hypothesis, at least if the bug is timing or a concurrency related. Some changes may change timing sufficiently that the bug hides. If the symptom disappears, it doesn't mean you solved the problem -- you must understand why and understand if the symptom disappeared or the bug get fixed. In one case as I fixed secondary bugs, the system stayed up longer and longer. But these are like accessories to the real murderer. You have to stay on the trail until you nail the real killer!
Another way of looking at this: a crime has been committed and since you don't know who the culprit is or where you may find evidence, you disturb the crime scene as little as possible, and restore things in case you have to move something.
But this is not usually what happens. People change things around without clear thinking -- change some code just because they don't like it or they think it can be improved or simplified -- and the symptom disappears and they declare success. Or they form a hypothesis, assume it is right and proceed to make the "fix" and if that doesn't work, they make another similar leap of logic. Or they fix a secondary issue, not the root cause so that the same bug will manifest again in a different place.
We don't need these things to be electives. We don't need them to be a master's program. We need an undergraduate software engineering program, and we need 90% of the people in CS to switch to that program instead.
The question as well is: are Chemical Engineering, Mechanical Engineering, Materials Engineering trade schools?
I think it's a key call out as CS touches on so many things.
There are arguments for it being math, science, engineering, a trade school or a combination of the above.
And then if you separate them out completely you end up with people doing CS being out of touch with what happens in the real world and vice versa.
I think in the end you probably need to have a single overall degree with a specialization in (Programming as Engineering, Programming as Math and Programming as Computer Science,) with lots of overlap in the core.
And then you still can have a both a bootcamp style trade school.
Now all of that said, that still doesn't solve the CS equivalent of "Math for business majors". Or the equivalent of "Programming for Scientists", or the like which is already a really important case to offer. Where you major in Bio/Chem/Other but being able to apply programming to your day job is important. Although that probably sits closer to the Applied Software category that you might find in business school like using spreadsheets, basic command lines, intro to databases or python.
But to your point, how rarely software engineering is being taught is a huge problem. Even if only 30% of degree holders took classes in it, it would be huge in helping spread best practices.
Software engineering is interesting, though, because it does not require as much formal education as many other engineering fields to get a job. I think this is in part because it is very economically valuable (in the US at least) and because the only tool you need is a computer with an Internet connection.
With all of that said, I think SWE is probably closer to a trade than other engineering disciplines, but not by all that much.
Which is a big shame, because it’s very common to see developers who would have benefited from a proper CS curriculum, and have to learn it later in life.
Software engineering isn’t just about teamwork, and not all software development-related teamwork skill is generalisable to other industries, but it’s far from uncommon for there to be some trendy blog post laying out the sorts of things that, yes, an MBA program will teach someone. Which is fine, if not for the fact that these same people will scoff at “clueless MBAs”.
After the first failed project, everyone agrees that they need a boss without assigned duties. Then in 5th year, we’d have one or two class-wide projects (20-30 people).
This, along with joining event-organizing charities on the campus, where you’d work 3-5hrs a week to build a hundred-thousand-to-a-million dollars event (NGOs on campus, or concerts, or a sports meeting, to each their preferences, but it was the culture). And it wasn’t always a good time together, it’s not flowers all the way down.
I’m only surprised some school might consider engineering without group effort. On the other hand, I deeply deeply regret spending this much time in charities and not enough dating, as I have never been recognized for charity service, and family was important to me.
This was a surprisingly effective course, if sadistic.
My spouse on the other hand took an explicitly IT program and they had group projects, engaging with real world users, building real solutions, etc.
If I had to guess, some polytechnic school or another?
With some classes even forbidding discussing work with other students, where each assignment required a signed (digitally or otherwise) affidavit listing everyone you consulted, acknowledging that if you actually listed anyone, you were admitting to violating the academic honesty policies and if you didn't list anyone yet had spoken with others you were of course also violating the academic honesty policies.
Where only consulting the professors or TAs was allowed, the TAs were never around, and the professors refused to help because if they gave you any hints, it would apparently give away the answer, which would be unfair to the other students.
So yeah, the guideline was that we'll be eventual judges of what was "too similar" and if you're concerned, just don't discuss implementation details with anyone. I realize it prevents honest colaboration, and that's bad too... but sometimes it's a "damned if you do, damned if you don't" kind of situation.
We caught 2 guys, of course we didn't know who copied from whom, but we quickly found out by challenging each of them with a fizz-buzz kind of question.
The best advice I got was to explain the difference between collaboration (strongly encouraged) and plagiarism (re-take the course if you're lucky). Forbidding collaboration is a disastrous policy, so an instructor shouldn't have a problem with giving the same mark to each member of a group of collaborators. You just have to test that each individual can explain what they've submitted.
FWIW, I was a "temporary visiting lecturer", i.e. contract fill-in help. I had to write the syllabus (literally), the course plan, and the hand-outs. I also had to write all the tests and the exams; and I had to mark the exams, and then sit on the exam board through the summer holidays. The pretty girls would flutter their eyelashes at me, and ask for leniency. I suspect they took their cue from my louche colleague, who was evidently happy to take sexual advantage of his students in exchange for better marks.
[Edit] I liked this colleague; but I would not have introduced him to my wife or my daughter.
Actually not so modern, my college was auto-grading 20 years ago.
(I certainly wasn't going to take digital submissions from them; that would mean floppy disks, and I regarded any disk that had been touched by any student as if it were infected with plague, because it almost certainly was. All the school systems were always infected).
One might think I'm just rating easier professors higher, but no, there were definitely enough sub-par teachers with lax policies and forgiving grading who failed to impart what their course was supposed to cover. There were also tough professors I learned a lot from. It's just that I can't recall a single decent instructor among those obsessed with anti-collaboration policies.
And I mean, that went with everything. In my EE classes we did lots of collaboration. We had study groups, homework groups, etc. It was a lot of fun. I'm sad to hear there are places where they straight up make that against the rules.
My uni also had engineering ethics classes - in my major it was mandatory to take 2 of them. I think it makes sense and should be more common for software engineers. A lot of software is used to operate planes, cars, medical equipment, and nowadays also help make decisions that can have life-threatening consequences.
Now I'm an adult, several years into my career, wishing that the schools had done a bigger quantity of low-stakes group projects instead. It's a muscle that was never exercised
As a learning tool it was a disaster. The code had never been touched by a skilled engineer, just years of undergrads pulling all-nighters.
That meant it didn't teach what good maintenance looked like. Students just piled their hacks on top of someone else's hacks so they could hit the deadline.
It wasn't a good way to learn to write quality software (even if reality sometimes looks this way).
It was also the only software engineering course in our CS curriculum.
This sounds oddly familiar... hmm.
What you're asking for is where vocational (2 year) programs really shine, and one of the few places where I've seen bootcamp graduates have an advantage. Unfortunately both then lack some really valuable underpinnings, example: relational algebra. It seems that once again there is no silver bullet for replacing time and experience.
It’s not the most defining class of my undergrad years, but it was pretty damned close.
The fact that most people designing systems don’t know this material inspires a mix of anger and existential dread.
It was practically an elective and these days I hope it’s required.
In my opinion, the intersection of cognitive psychology and software engineering is also ripe for a lot of research. I feel like we as software engineers have a lot of intuitions about how to be productive, but I would love to see various things that could indicate productivity measured in a controlled lab setting.
like the idea of a Poke Yoke could cover all manner of automated analysis software tools. from the -Wall in gcc to, Rust borrow checker, basically just Poke Yoke.
Computer science has changed more, there are lots of developments in the last 5-10 years.
So tell us what these are so I/we can learn from you
Software has changed in the last 10 years, but a lot of it has changed superficially. A green software engineer most likely won't be able to tell the difference between a superficial change and a fundamental change.
It has a lot to do with the topic of this thread. "Quality Software" It's a loaded term. There's no formal definition, everyone has their own opinion on it and even then these people with "opinions" can't even directly pinpoint what it is. So the whole industry just builds abstraction after abstraction without knowing whether the current abstraction is actually close to "quality" then the previous abstraction. It all starts out with someone feeling annoyed, then they decide to make a new thingie or library and then they find out that this new thing has new annoyances and the whole thing moves in a great flat circle.
That's the story of the entire industry just endless horizontal progress without ever knowing if we're getting better. A lot of the times we've gotten worse.
That being said there have been fundamental changes. Machine learning. This change is fundamental. But most people aren't referring to that here.
As opposed to maths, physics, philosophy, civil engineering, classical studies which have gone through complete revolutions in their topics, problems and study methods in the last 10 years?
Do you know if any of these courses has been opened sourced? It would be great to have access to part of the material.
The code base was for a simple service that basically provided virus/malware scanning and included the malware scanner and signatures (this ensured there would never be an end to the work - there's always more signatures to add, more features, etc.)
I thought this was a fantastic idea and its a pity he never convinced them. That was more than fifteen years ago, and in his plan it would have just run forever.
My experience has been that people who's first jobs were in companies with quality software or who's job included reading through other people's quality software learn to write good software, the other ones learn whatever they saw in the environments they worked in.
A lot of professors just want to do research and mentor students onto the PHD track to self replicate. My mandated faculty advisor was basically like "go to the career center" when I asked about, you know, getting a job of some sort with my degree.
So yes, it is a real problem. CMU may stand out by actually having courses in the space, but it is not the norm by any means.
This approach to education can work, but I think simulating/mocking portions of this would have been more helpful (it could've been a teacher/TA managed codebase we started with rather than the monstrosity passed between generations of students who were inexperienced.)
Learning why things are bad, and why it's bad to experience them offers a new level of appreciation or better ways to argue why certain things should be done.
We have that. Maybe not enough. But it's definitely not a new thing.
Surely, "it is taught", but to whom and how widely?
Ironically, the software engineering courses were the only ones I disliked while a student. An entire course in design patterns where strict adherence to UML was enforced felt a bit archaic. We had a course in software QA which mostly consisted of learning TDD and the standard tooling in the Java ecosystem, with some cursory walkthroughs of other types of QA like static analysis, fuzzing and performance testing. At the time it felt so boring, I liked to actually build stuff! A couple of years later I joined a team with very competent, CS-educated developers tasked to set up a workflow and all the tooling for testing software with high security requirements. They were dumbfounded when I knew what all the stuff they were tasked to do was!
What was not taught about were the techniques of QA's, the variety of tests and design pattern enforcements that would reduce QA cost by tenfolds.
When it comes to software engineering I feel there is no qualification where you can feel that the gross naivety about quality and working in teams (and with other teams) has been similarly ironed out.
Instead you get people hardened in sin from their repeated experience of writing one horrible bit of untested code quickly and then leaving before the shit truly hit the fan :-)
One's management is very impressed with those kind of people and very contemptuous of those left behind who have to painstakingly clean up the mess.
Parser tools are indeed very powerful, but those abstractions carry tradeoffs that any “engineer” will consider.
But I also wasn't focused on parser tools. My observation was more universal. That is, engineers look before they leap. Developers leap first and ask questions later. Engineers are intentional. Developers much less so, and far more reactive.
We can build something, or pull something off the shelf. If it takes x time to build it, and then x time to debug and test and x^(1/0) to maintain it; far better to just add a gem. Even if it's not the absolute best, at least it's easy to understand and if it becomes a problem fix the edges.
Maybe when writing your own programming language, own complicated data format or low communication protocoll while requiring extreme performance. But that seems to be super rare, at least in my area of profession.
Parser generators aren’t a free lunch and they vary considerably in quality.
In one case I gave up on fixing the code where every change introduced new possible bugs and used a parser generator. We never had another bug in that part of the code but my wholesale change caused intense friction.
I feel that a course in parsers would have helped that person to understand this wasn't an appropriate situation.
In fact I think it's a good idea to have BNF "that works" before you hand code anything just to confirm that you understand your own language design.
The only exception I personally know of is jq (uses bison). So it's difficult to produce helpful syntax error messages in the implementation of jq.
>>>
I consider the JetBrains parsing system to be world class and they seem to hand-write very few (instead building on this system: https://github.com/JetBrains/Grammar-Kit#readme )
- https://github.com/JetBrains/intellij-community/blob/idea/23... (the parser I'll concede, as they do seem to be hand-rolling that part)
- https://github.com/JetBrains/intellij-community/blob/idea/23... (same for its parser)
- https://github.com/JetBrains/intellij-community/blob/idea/23... and https://github.com/JetBrains/intellij-community/blob/idea/23...
- https://github.com/JetBrains/intellij-plugins/blob/idea/233.... and https://github.com/JetBrains/intellij-plugins/blob/idea/233....
Cryptography and parsers simply do not belong in the same sentence. There is never a time when it is a appropriate to write your own cryptography. OTOH, most large compiler and interpreter projects have handwritten parsers, and many of them have handwritten lexers too.
Writing a parser can be simple enough to fit into a take-home assignment, and hand-written parser code ends up looking pretty similar to an LL grammar anyway. Parsing is also the easiest part of writing compiler or language tooling, so if a hand-written parser is too high a bar for the team then the entire project might questionable.
I'm not saying never use a parser generator, but I would personally prefer to work on a project with a well tested hand-written parser than a project using a parser generator. Especially if it complicates the build process with extra tooling, or is something really dated like Bison or ANTLR.
Seems like a pretty bad premise to start an article on quality software. If you believe you can ship bug free code, it's time to switch careers.
Unfortunately, you are correct. Shipping in time and bug free are inversely proportional, and in a world were usually it's hard to argue with PMs for more time to have better testing, or paying tech debt... it's just a reality
But more importantly, once you've fixed the "show-stopping bugs," putting the software in front of customers is probably the best next step, as even if it's bug-free, that doesn't mean it solves the problem well.
[The above does not necessarily apply in highly regulated industries or where lives are on the line]
On those you're of course expected to do safety and testing up to the limit of the "value of a statistical life"s within the expected project impacts, but it still has time and budget limits.
Most software engineering is about making sure the happy path works well. When lives are on the line, you need to also plan to minimize the possible damage that can happen when things go wrong.
I can think of far better ways to spend infinite time.
We aren't taught how to write reliable software because very few people know how to write reliable software. It doesn't help that academia has a hard crush on OOP, which is a bag of accidental complexity - and complexity is a breeding ground for unreliability.
Ok, I think we’ve gone too far. There absolutely is such thing as 0 bugs and sometimes code changes don’t have bugs. That is not to say it can be garunteed.
Never said that, just that quick turnaround for deliveries will usually mean more bugs, and having some extra time usually means less bugs
I think when the author says "bug free", it's from the user perspective. where bugs either need to go out of your way to trigger or are so esoteric it's impossible to think about hitting without that user themself knowing the code inside out. Games is definitely an industry where the quality of code has always dipped to a point where users can easily hit issues in normal use, and only gets worse as games get more complex. That's where it gets truly intolerable
"Cheap. Fast. Good. Pick two."
The pessimistic viewpoint is that you get to pick up to one.
I assume I’m not infallible.
Writing some unit tests, C++ and mocking in my case, give both the team and myself some confidence I’ve not made things worse.
I’m the most experienced dev in the department.
Once you get beyond "I wrote a function" and "I tested a function" and even still "I tested a function that was called by a function over the wire", you will come to a realization that no matter how edgy your edge cases, no matter how thorough your QA, there will always - ALWAYS be 0-day "undefined behavior" in certain configurations. On certain hardware. On certain kernels. It's an assurance that I guarantee that I'm almost positive it's bug free, since it passed tests, it passed human eyes, and it passed review - fingers crossed.
Maybe show-stopping bugs are somewhat unique to software engineering, but all somewhat-complex products are flawed to some extent imho.
It might be an unergonomic handle on a frying pan, furniture that visibly warps under the slightest load (looking at ikea shelfing) or the lettering coming off the frequently used buttons on a coffee machine.
I’d also quibble about the buttons on the coffee machine. They might be properly designed, just subject to the normal wear-and-tear that is inevitable in the real world. This is not a defect, physical devices have finite lifespans.
As far as computers go… if we got to the point where the main thing that killed our programs was the hard drives falling apart and capacitors drying out, that would be quite impressive and I think everyone would be a little bit less critical of the field.
As the discipline has evolved, the high integrity tools are slowly being incorporated into typical languages and IDEs to generally improve quality cheaper. Compare C++ to rust for example, whole classes of bugs are impossible (or much harder to make) in rust.
And this is almost the simplest possible program.
But I mean, I basically agree that the ecosystem is too complicated.
New bridges collapses, dams overflow s, planes crashes, vaccines kills, food kills, leaning towers and skyscrapers, capsized ships - catastrophic flaws are everywhere.
There's a saying that I think fits very well here: "Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands."
You don't want a bridge to cost 50 years and quadrillions of dollars to build, you want a cheap bridge safe for the next 50 years done in 2 years.
I would not call the resulting bridge "bug free", of course.
Bugs arise from the inherent complexity introduced by writing code, and our inability to foresee all the logical paths a machine can take. If we're disciplined, we write more code to test the scenarios we can think of, which is an extremely arduous process, that even with the most thorough testing practices (e.g. SQLite) still can't produce failproof software. This is partly because, while we can control our own software to a certain degree, we have no control over the inputs it receives and all of its combinations, nor over the environment it runs in, which is also built by other humans, and has its own set of bugs. The fact modern computing works at all is nothing short of remarkable.
But I'm optimistic about AI doing much better. Not the general pattern matching models we use today, though these are still helpful with chore tasks, as a reference tool, and will continue to improve in ways that help us write less bugs, with less effort. But eventually, AI will be able to evaluate all possible branches of execution, and arrive at the solution with the least probability of failing. Once it also controls the environment the software runs in and its inputs, it will be able to modify all of these variables to produce the desired outcome. There won't be a large demand for human-written software once this happens. We might even ban software by humans from being used in critical environments, just like we'll ban humans from driving cars on public roads. We'll probably find the lower quality and bugs amusing and charming, so there will be some demand for this type of software, but it will be written by hobbyists and enjoyed by a niche audience.
People who know how the sausage is made will always know of a bunch of bugs that haven't been fixed exactly because they aren't impactful enough to be worth the effort required to fix them.
The other is working out the correct specification is far harder than coding is.
For example it is trivial to write a bug free program that multiplies an integer between 3 and 45 by two.
No, it's functional. If it has bugs, it's not bug-free. By definition.
To quote a former marketing guy “it should work out what the user intends to do and do it”?
Not sure what your quote is supposed to mean. That's a textbook example of someone who doesn't understand software at all making laughable requests of their engineers.
The reason for that quote is from what you have said a bug would be anything you didn't expect, even if it is consistent or not with the specification as that merely affects if we classify it as functional or not (a classification I profoundly disagree with, obviously). It is simply a negative rephrasing of what the marketing guy said and laughable in the same way.
> One plausible definition is “system deviates from its specification”
And that's quite reasonable. So I actually retract my argument.
For my own definition, I was considering a bug to be any behavior that the software engineers weren't expecting. Because those can exist invisibly for a long time until they become so bad they become visible. They can also exist for decades without causing any problems to functionality at all.
Really bug free commercial software does not exist. And can't exist. There are always bugs which are known but would not be fixed.
You would have to introduce things like tolerance to hardware failure, but that is outside the spec of the software as stated.
In the same way I am not a real Scotsman, your toy example of an easily specified snippet of a function that doesn't do anything useful is not real software.
No-one in the real world gives a damn about your 'spec of the software as stated'
When testing should start? BEFORE the first line of code is written.
(Personally, I think code is itself more analogous to the specification artifacts of other professions - eg. blueprints - and the process of creating the machine code of what is analogous to construction / manufacturing something to those specs.)
And even having said that, even the vast majority "bug free" software that nearly always appears to be operating "within spec" will have corner cases that are expressed in very rare situations.
But none of this is an argument for nihilism about quality! It is just not the right expectation going into a career in software that you'll be able to make things that are truly perfect. I have seen many people struggle with that expectation mismatch and get lost down rabbit holes of analysis paralysis and overengineering because of it.
Except for ones that do, obviously.
The key reason to make the distinction is because the fuzzy business of translating intention into specification needs to be fully accepted as an ongoing negotiation process of defining exactly what the specification is, and integrated into repeated deterministic verification that that is what has been delivered. Failing to do that is mainly a great way for certain software management structures to manipulate people by ensuring everything is negotiable all the time, and has the side effect that no one can even say if something is a bug or not. (And this pattern is very clear in the discussion in this thread - there is a definite unwillingness to define what a bug is).
IME the process of automated fuzzing radically improves all round quality simply because it shakes out so many of the implicit assumptions and forces you specify the exact expected results. The simple truth is most people are too lazy and/or lack the discipline needed to do it.
Nobody actually has any trouble knowing what a bug is. Like, this is just a non-issue, I've never in my career spent a non-negligible amount of time debating with anybody whether something is or isn't a bug. We discuss whether fixing bugs have worthwhile return on investment, and we discuss the relative priority between fixing bugs and doing other things, but this meta-debate about "well technically it complies with this spec so is it even a bug, really?" just never comes up. We all know what bugs are.
I have been writing code for almost a decade now, and still make errors. I don't believe anyone is capable of producing bug free software.
I have also seen plenty of bugs in apps and games. I don't think I have ever witnessed a major game patch that was bug free.
Microwave has random errors from time to time.
Robo vacuum freezes.
Parking meter malfunction.
Public transport ticket machine don't want to give me a ticket.
Online banking failing to make a transfer because I use UI with other language.
Mobile banking failing to make a transfer because I use not native currency.
Car has issues as well, incorrect fuel amount is injected by computer.
Online pages have tons of bugs, many are barely usable.
Most people in this thread understand that users' interests only matter insofar as they impact business profit.
I just think you're having a different conversation.
Fact is, if you ship enough 'low probability' bugs in your product, your probabilities still add up to a point where many customers are going to hit several of them.
I've used plenty of products that suffer from 'death by a thousands cuts'. Are the bugs I hit "ship blockers"? No. Do I hit enough of them that the product sucks and I don't want to use it? Absolutely.
Right, you don't want to use Microsoft Word, or SalesForce, or Apple vs Android, or X Whatever. It's highly unlikely you'll have a choice if you use it though.
If you're in a very large FANNG type company, and say you have 1000 components that each ship 1 bug each day that has a 0.1% chance of breaking something critical, that translates to a less than 50% chance you ship a working OS on any given day. And that may mean the entire company's productivity is impacted for the day depending on how broken it is.
"A programmer who releases buggy software and fixes them is better than a programmer who always releases perfect software in one shot, because the latter doesn't know how to fix bugs."
Perhaps similar to the saying that a good driver will miss a turn, but a bad driver never misses one.
The doesn't know how to fix bugs idea seems pretty silly.
The moral of the sayings is, that "perfect" programmer is actually a bad programmer because he wouldn't know how to fix bugs by virtue of never needing to deal with them.
To reuse the driver analogy, the driver who never misses a turn is a bad driver because he doesn't know what to do when he does miss a turn.
If a software developer consistently delivers high-quality software on time and on budget, that means they're good at their job, pretty much by definition. It would make no sense to infer they're bad at fixing bugs.
It would make sense to infer instead that they're good at catching and fixing bugs prior to release, which is what we want from a software development process.
> the driver who never misses a turn is a bad driver because he doesn't know what to do when he does miss a turn
Missing a turn during a driving test will never improve your odds of passing.
The driver who never misses a turn presumably has excellent awareness and will be well equipped to deal with a mistake should they make one. They also probably got that way by missing plenty of turns when they were less experienced.
What we are discussing isn't a real programmer we might actually find. No, we are talking about a hypothetical "perfect" programmer. This "perfect" programmer never wrote a bug in his entire life right from the moment he was born, he never had a "when they were less experienced" phase.
Obviously, that means this "perfect" programmer also never debugged anything. For all the perfect code he writes, that makes him worse than a programmer who writes buggy code but also knows how to go about debugging them.
Whenever I hear a developer say "I never ship buggy code", I am always cautious to dig in more and understand what they mean by that.
I jest, but that should be the gold standard for anything life-critical and good to have for mission-critical software. Alas, we're not there yet.
There are well-formed statements that can be proved but which assert that its godelized value represents a non-provable theorem.
Therefore, you must accept that it and its contradiction are both provable (leading to an inconsistent system), or not accept it and now there are provable theorems that cannot be expressed in the system.
Furthermore, that this can be constructed from anything with base arithmetic and induction over first-order logic (Gödel's original paper included how broadly it could be applied to basically every logical system).
And, since many proofs are done by assuming a statement's inverse and trying to prove a contradiction, having a known contradiction in the set of provable statements can effectively allow any statement to be proven. Keeping the contradiction is not actually an option.
Godel means that we can't have an algorithmic box that we put a program into and out comes a true/false statement of halting.
Nothing is stopping you from writing the proof manually for each program that you want to prove properties for.
ALSO, you can write sub-turing complete programs. Those are allowed to have automated halting proofs (see idris et al).
While, yes, computability and provability are very closely related, it's important to get attribution correct.
More details on what Gödel's Incompleteness Theorem really said are in a sibling comment so I won't repeat them here.
Really? Says who?
Or perhaps you'll prove it from first principles. Although if turns out to be difficult, that's okay. Somebody mentioned something about systems being either complete or consistent but never both. Some things can be true but not proveably so. Can't quite remember who it was though.
[I believe that] it's important to get attribution correct.
- Futurama (kind of)
Gödel's really was a rather unique mind, and the story of his death is kind of sad.. but I wonder if it takes such a severe kind of paranoia to look for how math can break itself, especially during that time when all the greatest mathematicians were in pursuit of formalizing a complete and consistent mathematics.
It might be wrong, but it's pretty easy to inspect and has a much higher chance of being right than your code does.
You then use proof refinement to eventually link this very high level statement down to the code implementing it.
That's the vision, at least, and it's sometimes possible to achieve it. See, for example, Ironfleet: https://www.microsoft.com/en-us/research/publication/ironfle...
Bugs in the formal spec aren't impossible, but use of formal methods doesn't prevent you from doing acceptance testing as well. In practice, there's a whole methodology at work, not just blind trust in the formal spec.
Software developed using formal methods is generally assured to be free of runtime errors at the level of the target language (divide-by-zero, dereferencing NULL, out-of-bounds array access, etc). This is a pretty significant advantage, and applies even if there's a bug in the spec.
Disclaimer: I'm very much not an expert.
Interesting reading:
* An interesting case-study, albeit from a non-impartial source [PDF] https://www.adacore.com/uploads/downloads/Tokeneer_Report.pd...
* An introduction to the Event-B formal modelling method [PDF] https://www.southampton.ac.uk/~tsh2n14/publications/chapters...
Yes, with the current state of the art, adopting formal methods means adopting a radically different approach to software development. For 'rapid application development' work, it isn't going to be a good choice. It's only a real consideration if you're serious about developing ultra-low-defect software (to use a term from the AdaCore folks).
> it doesn't even eliminate bugs because the formal proof could include a bug too
This is rather dismissive. Formal methods have been successfully used in various life-critical software systems, such as medical equipment and avionics.
As I said above, formal methods can eliminate all 'runtime errors' (like out-of-bounds array access), and there's a lot of power in formally guaranteeing that the model's invariants are never broken.
> I suppose it could help in trivial cases like sanity checking that a value isn't null or that a lock is only held by a single thread
No, this doesn't accurately reflect how formal methods work. I suggest taking a look at the PDFs I linked above. For one thing, formal modelling is not done using a programming language.
If you can't use formal proof just because the user can't be arsed to wait where it is supposed to be necessary, then the software project conception is simply not well designed.
This is the fallacy people have when thinking they can "prove" anything useful with formal systems. Code is _already_ a kind formal specification of program behavior. For example `printf("Hello world");` is a specification of a program that prints hello world. And we already have an abundance of tooling for applying all kind of abstractions imaginable to code. Any success at "proving" correctness using formal methods can probably be transformed into a way to write programs that ensure correctness. For example, Rust has pretty much done so for a large class of bugs prevalent in C/C++.
The mathematician's wet dream of applying "mathematical proof" on computer code will not work. That said, the approach of inventing better abstractions and making it hard if not impossible for the programmer to write the wrong thing (as in Rust) is likely the way forward. I'd argue the Rust approach is in a very real way equivalent to a formal specification of program behavior that ensures the program does not have the various bugs that plagues C/C++.
Of course, as long as the programming language is Turing Complete you can't make it impossible for the programmer to mistakenly write something they didn't intend. No amount of formalism can prevent a programmer from writing `printf("hello word")` when they intended "hello world". Computers _already_ "do what I say", and "do what I mean" is impossible unless people invent a way for minds to telepathically transmit their intentions (by this point you'd have to wonder whether the intention is the conscious one or the subconscious ones).
As I already said in my reply to xmprt, formal methods have been used successfully in developing life-critical code, although it remains a tiny niche. (It's a lot of work, so it's only worth it for that kind of code.) Google should turn up some examples.
> Code is _already_ a kind formal specification of program behavior.
Not really. Few languages even have an unambiguous language-definition spec. The behaviour of C code may vary between different standards-compliant compilers/platforms, for example.
It's possible to reason formally about C, but it's not an ideal match. https://www.eschertech.com/products/ecv.php
The SPARK Ada language, on the other hand, is unambiguous and is amenable to formal reasoning. That's by careful design, and it's pretty unique. It's also an extremely minimal language.
> `printf("Hello world");` is a specification of a program that prints hello world
There's more to the story even here. Reasoning precisely about printf isn't as trivial as it appears. It will attempt to print Hello world in a character-encoding determined by the compiler/platform, not by the C standard. It will fail if the stdout pipe is closed or if it runs into other trouble. Even a printf call has plenty of complexity we tend to just ignore in day to day programming, see https://www.gnu.org/ghm/2011/paris/slides/jim-meyering-goodb...
> Any success at "proving" correctness using formal methods can probably be transformed into a way to write programs that ensure correctness
You've roughly described SPARK Ada's higher 'assurance levels', where each function and procedure has not only an ordinary body, written in SPARK Ada, but also a formal specification.
SPARK is pretty challenging to use, and there can be practical limitations on what properties can be proved with today's provers, but still, it is already a reality.
> Rust has pretty much done so for a large class of bugs prevalent in C/C++
Most modern languages improve upon the appalling lack of safety in C and C++. You're right that Rust (in particular the Safe Rust subset) does a much better job than most, and is showing a lot of success in its safety features. Programs written in Safe Rust don't have memory safety bugs, which is a tremendous improvement on C and C++, and it manages this without a garbage collector. Rust doesn't really lend itself to formal reasoning though, it doesn't even have a proper language spec.
> The mathematician's wet dream of applying "mathematical proof" on computer code will not work
Again, formal methods aren't hypothetical.
> I'd argue the Rust approach is in a very real way equivalent to a formal specification of program behavior that ensures the program does not have the various bugs that plagues C/C++.
It is not. Safe languages offer rock-solid guarantees that certain kinds of bugs can't occur, yes, and that's very powerful, but is not equivalent to full formal verification.
It's great to eliminate whole classes of bugs relating to initialization, concurrency, types, and object lifetime. That doesn't verify the specific behaviour of the program, though.
> No amount of formalism can prevent a programmer from writing `printf("hello word")` when they intended "hello world"
That comes down to the question of how do you get the model right? See the first PDF I linked above. The software development process won't blindly trust the model. Bugs in the model are possible but it seems like in practice it's uncommon for them to go unnoticed for long, and they are not a showstopper for using formal methods to develop ultra-low-defect software in practice.
> "do what I mean" is impossible unless people invent a way for minds to telepathically transmit their intention
It's not clear what your point is here. No software development methodology can operate without a team that understands the requirements, and has the necessary contact with the requirements-setting customer, and domain experts, etc.
I suggest taking a look at both the PDFs I linked above, by way of an introduction to what formal methods are and how they can be used. (The Formal methods article on Wikipedia is regrettably rather dry.)
But right from the outset the approach is doomed to fail because its proponents write like they don't know what they are talking about and think they can write bug-free software.
It really should be "write software with a formal spec". Once people start talking about "proof" in practice it sounds dishonest. It isn't possible to prove software and the focus really needs to be on the spec.
The code is already a formal spec.
Unless there are bugs in the language/compiler/interpreter, what the code is essentially formally well defined.
As programming languages get better at enabling programmers to communicate intention as opposed to being a way to generate computer instructions, there's really no need for a separate "spec". Any so called "spec" that is not a programming language is likely not "formal" in the sense that the behavior is unambiguously well defined.
Of course, you might be able to write the "spec" using a formal language that cannot be transformed into machine code, but assuming that the "spec" is actually well defined, then it's just that "compiling" the spec into machine code is too expensive in some way (eg. nobody has written a compiler, it's too computationally hard to deduce the actual intention even though it's well defined, etc.). But in essence it is still a "programming language", just one without a compiler/interpreter.
You can formally prove that it doesn't have certain kinds of bugs. And that's good! But it also is an enormous amount of work. And so, even for life-critical software, the vast majority is not formally proven, because we want more software than we can afford to formally prove.
Proving that the program has no bugs is akin to proving that the program won't make you feel sad. Like ... I'm not sure we have the math.
One of the more important jobs of the software engineer is to look deep into your customer's dreams and determine how those dreams will ultimately make your customer sad unless there's some sort of intervention before you finish the implementation.
What actually would solve the customer's problem -> What the customer thinks they want -> What they communicate that they want -> What the requirements collector hears -> What the requirements collector documents -> How the implementor interprets the requirements -> What the implementor designs/plans -> What the implementor implements.
Formal proofs can help with the last 3 steps. But again that's assuming the implementor can formalize every requirement they interpreted. And that's impossible as well, there will always be implicit assumptions about the running environment, performance, scale, the behavior of dependent processes/APIs.
It helps with a small set of possible problems. If those problems are mission-critical then absolutely tackle them, but there will never be a situation where it can help with the first 5 steps of the problem, or with the implicit items in the 6th step above.
Formal proving doesn’t really scale much further, definitely not to “enterprise” product scale.
Usually it's something extremely simple to fix too.
No kidding. I'm 2 or 3 years into working on a SaaS app started in ~2013 and I still get bug reports from users that make me say "what!? we have that feature!?"
The difference is that software can be used before it is fully ready, and it makes sense to do so. No one can really use a 90% finished power plant, but software at 95% capacity is still usually "good enough"
The end result is never 100% perfect, but somewhere in between "not too bad" and "good enough".
If you want to patch a bridge, it's gonna cost you. Even if you only need to close down a single lane of traffic for a few hours you are looking at massive expenses for traffic control, coordination with transportation agencies, etc.
For most software it's pretty inexpensive to ship updates. If you're a SaaS company regular updates are just part of your business model. So the software is never actually done. We just keep patching and patching.
In some contexts, it is much more expensive to push out updates. For example, in the 00s, I worked on a project that had weather sensors installed in remote locations in various countries and the only way to get new software to them was via dial-up. And we were luck that that was even an option. Making international long distance calls to upload software patches over a 9600 baud connection is expensive. So we tested our code religiously before even considering an update, and we only pushed out the most direly needed patches.
Working on SaaS these days and the approach is "roll forward through bugs". It just makes more economic sense with the cost structures in this business.
There's even an added factor of learning more about what is really needed by putting a 95% done product into use.
Heck, it explains (stretching it here) space-x's success with an iterative approach to rocket design.
No, you just complain that your taxes are being used to build expensive roads and bridges. Or you think airplanes are far too expensive. Or that new cars are insanely expensive.
There are cost trade offs. In general, better quality more expense.
Also in software there is not an excessive amount of software engineers in relation to demand for software. So SWEs can get paid a lot to go build crappy software.
More importantly though, for the most part we choose not to do engineering. By which I mean this - we know how to do this better, and we apply those techniques in areas where the consequences of failure are high. Aerospace, medical devices, etc.
It differs a bit industry to industry, but overall the lessons are the same. On the whole it a) looks a lot more like "typical" engineering than most software development and b) it is more expensive and slower.
Overall, we seem to have collectively decided we are fine with flakier software that delivers new and more complex things faster, except where errors tend to kill people or expensive machines without intending to.
The other contributing thing is it's typically vastly cheaper to fix software errors after the fact than, say, bridges.
1. The ECU ("hard" engineering)
2. The infotainment system ("soft" engineering)
Now, an interesting thing I have noticed is that "soft" software engineering pays more. Often substantially more.
Further to that, it's often informative to figure out what makes a company money. The highest paid software development roles tend to be doing things that are closer to revenue, on average. If you are a software developer at a hardware company (or an insurance company, or whatever), you aren't that close. Even worse if you are viewed as a cost center.
yeah. Who are those trillion dollar businesses and what do they rely on?
- Apple: Probably the better example here since they focus a lot on user-facing value. But I'm sure they have their own deals, B2B market in certain industries, R&D, and ads to take into account
- Microsoft: a dominant software house in nearly every aspect of the industry. But I wager most of their money comes not from users but other businesses. Virtually every other companies uses Windows, Word, and those that don't may still use Azure for servers.
- Alphabet: ads. Need I say more? Users aren't the audience, they are the selling point to other companies.
- Amazon: a big user facing market, but again similar to Microsoft. The real money is b2b servers.
- Nvidia: Again, user facing products but the real selling point is to companies that need their hardware. In this case, a good 80% of general computing manufacturers.
- Meta: Ads ans selling user data once again
- Tesla: CEO politics aside, it's probably the 2nd best example. Split bewteen a user facing product that disrupted an industry and becoming a standard for fuel in the industry they disrupted. There's also some tangential products that shouldn't be underestimated, but overall a lot of value seems to come from serving the user.
General lesson here is that b2b and ads are the real money makers. if you're one level removed that financial value drops immensely (but not necessarily to infeasible levels, far from it).
That sounds like an awfully bold claim. I have the feeling we understand software a lot better than we understand mechanical engineering (and by extension material sciences) or fluid dynamics. By a big margin.
I worked with finite element software and with CFD solvers, you wouldn't believe how hard it is to simulate a proper airflow over a simple airfoil and get the same results as in the wind tunnel.
To the contrary, it's nearly canonical. Most of the problems pointed out in the 70s (mythical man month) have still not been resolved, 50 years later.
>you wouldn't believe how hard it
Oh, I'd believe it (I've designed and built similar things, and had colleagues in CFD).
But you are definitely cherry picking here. The problem with CFD is we don't understand the fluid dynamics part very well; turbulence is a big unsolved problem still, though we have been generating better techniques. This is so true that in an undergraduate physics degree, there is usually a point where they say something like: "now that you think you know how lots of things work, let's introduce turbulence"
But a lot of mechanical engineering and the underlying physics and materials science is actually pretty well understood, to the degree that we can be much more predictive about the trade offs than typically is possible in software. Same goes for electrical, civil, and chem. Each of them have areas of fuzziness, but also a pretty solid core.
Of course it is easier for a developer to walk away from something incomplete than an architect and the contractors involved in a physical project, but still, I hardly think that there is really much difference in terms of timelines.
Software is still absolutely king for number of large scale projects that just never ship, or ship but never work.
Even with all of those applied, we wouldn’t be magically better. Complexity is simply unbounded. It’s almost impossible to reason about parallel code with shared mutable state.
Team are flying the airplane, the se time rebuild it to the zeppelin, testing new engines inflight.
Or construction. Let's build apartment block, but for few apartments we will test new materials, new layout, etc. Once there are walls of the first apartments we will let people live there. We will build how we can, according to the napkin plan. In the end we will put all tenants in and stress test strength of the structures. Or one day people return home and their apartments have totally different design and layout because someone from the HOA decided so to get a promotion.
Now we don't accept an engineer saying, "this bridge will probably collapse without warning", which we do accept with software. So there is a difference.
Building a house will have a restrictive initial budget for complexity, you don’t have enough in that budget for rotating floors, or an elevator that is catapulted to the correct floor, etc. These would cost both at engineering time and implementation time a huge amount. Less complexity is easier to analyze.
In case of software, complexity has negligible cost, relative to physical systems. You can increase it ad infinity — but proving it (the whole stack - from the hardware-OS-userspace software) correct is likely impossible with even the whole of mathematics, in certain cases.
ive had this sentiment thrown at me too often by peak move fast and break things types. it's too often a cudgel to dispense with all QA in favor of more new feature development. shipping shit that has the same pattern of flaws youve encountered in the past when youve been shown ways to catch them early but couldnt be bothered isnt accepting that you cant catch everything, it's creating a negative externality.
you usually can make it someone else's problem and abscond with the profits despite, but that doesn't mean you should
Not buildings, not bridges, not cars, not airplanes, not software. There are mistakes in every field of engineering and the best we can hope for is to minimize them as much as possible.
Engineering is knowing (among other things) how to categorize the potential mistakes, develop procedures to reduce the chance of them being made and in the case that some slip through (and they will), estimate their impact and decide when you're "good enough."
Software engineering is no different.
Currently ways of coding to this kind of standard exist. But they are stupid. It involves things like no dynamic memory allocation, only fixed length for-loops, and other very strict rules. These are used in aerospace, where bugs are rather costly and rare.
Engineers cannot be responsible for all the vertical stack and the components which were built by others. If somebody claims it is bug free then they have not enough experience. Pretty much anything can fail, we just need to test as many possible cases as possible with a variety of available tools to reduce the chances of bugs.
It is a good peek into the professional software world though!
Writing a thousand lines of bug free code isn't that hard, so the need for QA practices won't be apparent. Then you get the ten thousand line intern project and discover things are not always that easy. Then we throw you into a multi-million line project and good luck: you need a lot of that QA stuff to make it work.
A pure university education without considering is this degree useful in the real world is a disservice to education. However a pure trade school education that teaches how to do something without understanding is not useful (I don't think any trade school is that pure: they tell you to ignore hard stuff but generally give you deep understanding of some important things)
Why?
> A pure university education without considering is this degree useful in the real world is a disservice to education.
I think this line of thinking is a much bigger disservice to higher education. It was very tiresome as an undergraduate to be surrounded by people that thought this way - and detrimental to everyone's education.
"I'll never use this knowledge" is the single worst thing you can say as a student, and it needs to be beaten out of undergrads' heads. Not encouraged.
Which is why the "I'm never going to use this, what a waste of time" feeling among American undergrad students is so common. If you fix the affordability problem and bring it back to where is was in the mid 70s (inflation adjusted) I think things would be a lot better.
That was true 50 years ago, but employers turned it into job training. My father in law retired a well off businessman with a History degree from Yale he got in the 50s. You know what a History degree from Yale qualifies you for today? Teaching History and maybe writing some books. The degree didn't change and Yale didn't change.
I'm not sure about this part... A very common pattern in my conversations with working class friends and family from my parents' generation is: "we were told that if we sent our kids to college, they'd have better lives than we did, but instead we all just ended up with more debt than we could handle".
It's tricky! If you tell teenagers and their parents the truth - this purely academic program will not train you for any job besides pure academia, which, while it can be a fantastic career, is a super risky hits business in which only a few will truly succeed - then that's only going to sound like a reasonable risk to take for wealthy families. But then you've badly limited your pool of academic researchers to this extremely small and honestly often not as promising set of rich kids.
Maybe one solution (which is not workable in the real world) would be: any academic program that does not have a viable "job training" component should only accept students on academic scholarship, regardless of their own means. If some neutral party thinks they are promising enough in that field to pay their way, they get to go for free, otherwise they don't get to go at all. The programs that do graduate people with directly marketable job skills could keep working the current mercenary way.
The reason this wouldn't work in reality is that the wealthy would still just game the scholarships in some way. Alas.
There is nothing wrong with art/music/history. If you are interested by all means take a lot of courses in them. You can learn a lot of valuable skills which is why good universities required a diverse background of "generals" that these (and many more) fit into. However they give far more degrees in these things than are needed. (even physics gets more degrees than the world needs - but most getting a physics degree can better pivot to something else well paying).
If you want to know what a university will teach your kids, ask them. They'll even tell you without asking them - it was pretty obvious to me as a dumb high school kid on campus visits what the emphasis of one program or another was going to be.
If you are a working class family with a kid who is very talented at math, and you go sit down with the counselors and ask them: If my child studies pure theoretical math, will that open them up to a life full of possibilities? they will say "yes, it absolutely will". But that's not true. It might be true, but it's a big risk. It's a risk a wealthy family can very easily absorb. But if this child from this working class family takes on this risk using student debt, it might go poorly. They might very well be good at pure math but not be good enough to go into academia. Then they might be unsure what else they can do with that degree, unable to get their foot in the door at the kinds of employers where just a general proof-of-being-smart degree is enough. And now they have debt and uncertainty about what to do.
It also might work out great! But it's a risk. And I know a number of people who feel they ended up on the wrong side of that risk.
99% of what we all learn in college is a waste of time for us. But we all have a unique 1% that is vital to who we become. Over time I expect that 1% to become 0.1%, then 0.01%, and for that vitality to become ever more concentrated in that sliver.
Sure there are a few going to university just for the fun of it. However most are expecting a job. Thus universities should train and emphasize thinking in more specific areas.
> "I'll never use this knowledge" is the single worst thing you can say as a student, and it needs to be beaten out of undergrads' heads. Not encouraged.
This is tricky. I agree undergrads say this all the time when they are wrong but they don't know it. They have no clue what they will use and what they won't. This is something universities should figure out so they push people to avoid things they won't use. OTOH, a lot of what they are really teaching isn't the specific skill, but how to research and analyze data to find complex answers - it doesn't matter if you look at data from art or from science, what you are really learning is how to think and the specific knowledge gained is isn't important or the point (I think this is the point you were trying to make?).
Companies that offer the jobs are the ones that need to offer the job training.
> (I think this is the point you were trying to make?)
Not really, it's that university education is kind of meta/self serving (the goal is not to train X number of students to do Y jobs, it's to give every student at the institution what that institution defines to be an education).
But like you said, the fact this produces better workers is a second-order effect. It's not the goal of most institutions. But not all institutions; some define "well educated" to have lots of industry practicum, and if you want that, go study at those institutions.
My main point is that it's not a "disservice" to eschew practicum or industry training as an educational institution.
When institutions allow student to take degrees that society finds less valuable (art,music...) they are doing society a disservice by not producing what society needs. Now if the student is wealthy (not rich) enough to afford that the price then I don't care: I don't need to impose my values on anyone else. However most people in a university are not that wealthy (most are young) and so if the degree granted isn't valuable to society the university robbed that student.
1. what's wrong with a student pursuing their own personal goals? A person doesn't need to produce for society's sake.
2. despite that sentiment you hold, it's clear many people do value art and music. Maybe not in its pure form, but those artists do in fact fuel industries worth billions. Clearly "society" values something that requires such skills and thinking.
No. Companies love hiring higher-ed graduates because it removes a lot of cost and risk for them:
- hiring only people with degrees weeds out everyone unable to cope with a high-stress environment, for whatever reason - crucially, also including people who would normally be protected by ADA or its equivalent provisions in Europe.
- it weeds out people in relationships or with (young) children, which makes them easier to exploit and reduces the amount of unexpected time-off due to whatever bug is currently sweeping through kindergarten/school/whatever. Sure, eventually they will get into relationships and have children as they age, but looking at the age people start to have kids these days [0], that's a solid 5-10 years you can squeeze them for overtime.
- it saves companies a ridiculous amount of training. The old "tradespeople apprenticeship" way is very cost-intensive as you have to train them on virtually anything, not just stuff relevant to the job, e.g. using computers and common office software. Instead, the cost is picked up either by the taxpayer (in Europe) or by the students themselves in the form of college debt. The latter used to be reserved for high-paying jobs such as pilots who have to "work off" their training cost but got compensated really well, nowadays it's standard practice.
- it keeps the employee diversity relatively homogenous. There is a clear bias towards white and asian ethnicity in the US for higher ed [1], and among top-earning job, males still utterly dominate [2].
- related to the above, it also weeds out people from lower economic classes, although at least that trend has been seriously diminishing over the last decades [3].
[0] https://www.nytimes.com/interactive/2018/08/04/upshot/up-bir...
[1] https://hechingerreport.org/proof-points-new-higher-ed-data-...
[2] https://www.bankrate.com/loans/student-loans/top-paying-coll...
[3] https://www.pewresearch.org/social-trends/2019/05/22/a-risin...
And thus you destroyed you whole point: females dominate universities these days.
[1] https://www.stemwomen.com/women-in-stem-percentages-of-women...
Unfortunately many programs are not designed this way and learning the appropriate skills is left as an exercise to the student usually in a sink or swim approach. So some students come out with the meta skills that a university education is touted for and others do not.
I do agree that "I'll never use this knowledge" can be a miserable attitude to have or engage with - especially when it's just a proxy for "I'm not interested in learning, just in getting good grades" but the idea itself is valid.
Everyone will think differently. I've never truly be research-minded and there's very much a bunch of odd classes that felt like a waste of my money (something to consider as education gets more expensive). But I do agree that there should be a space to foster researchers and especially one to overall round out a student, even if that space is more niche. I just don't think that everyone needs to go far into debt for that experience if they just want job training.
So I too desire a more explicit divide than "research university vs. industry university" and wish there were some better trade schools focused on software (not 6 month boot-camps. Think of a condensed university program without requirements of electives and maybe less supporting classes). But no one seems to be protesting this much.
Today? TailScale and stuff like that.
You can just noT have a bunch of pointless micro services and docker in your runc and layer on layer of json de/re, and unit test to get coverage but ignore quickcheck and hypothesis and fuzzing.
You can use stacked diffs and run oncall loops out of the team authoring the code and all of it.
You can minimize dynamic linking and all the other forms of unforced dependency error.
You can understand and play towards the runtimes of all the managed languages in your stack. You can insist that a halfway decent verbal grammar for the languages is “readability”.
It gets shouted down over and over but it’s public knowledge how to ship quality software.
If you are doing network switches or GPUs or server CPUs or whatever, yeah, technology matters. If you are building pretty much any SaaS, MCCA, etc. the tech is literally irrelevant and mostly the more "new" tech you use the worse off you are.
Quality also only matters in some contexts and those are even rarer than the above.
Timing and solving a problem is all that matters in terms of revenue. As long as you aren't too bad, it's fine. The only real five-nines requirement for SaaS is that 99.999% of them would be fine with one or two nines.
It then looks like it was a "business reason" in the end, but it's not so easy to just distinguish what the root cause really was.
What does roughly FAANG mean?
Today? TailScale and stuff like that.
The VPN?
You might have some ideas about quality software, but this comment is incomprehensible.
Yeah you won't get to a point where you'll have a valid-enough metric to make this point.
I was at a startup once. The two founders said "don't write unit tests". I wasn't going to argue with them. I understood what they really meant. We've been too slow, we need to ship as fast as possible. I shipped fast and I shipped quality (ie low defects and outages). I wrote unit tests. They didn't need to know. They just needed the outcome.
The elephant in the room in all of these conversations is that you walk into any software development shop and they just don't how to ship both fast and at quality. No matter how much an organization or team tries to revisit/refactor their dev process year-to-year, they're still shipping too slow and mediocre quality.
The truth is there isn't a magic formula. It's an individual craft that gets you there. It's a team sport. And the context is different enough everywhere you go, yeah, sure, some lightweight processes might abstract across the universe but very little bang for those bucks. Far beyond any other facet of things, you really just have to have a good team with experience and the right value-system/value-delivery-focused wisdom.
2. Is this article from the 90s where we ship software on CDs or floppy disks? In today's world where the concept of a "release" is often blurred by continuous delivery pipelines (and this is considered a good practice), having a quality insurance department manually assuring that no bugs are in that release seems downright archaic.
Anything connected to a network could be released frequently if people wanted to. Not everything is connected to a network though.
Some things should not roll out updates without extensive testing, including manual testing to verify nothing was missed.
There are tons of software without continuous delivery.
The more general way of saying this is, "If you need someone's approval to do something, explain it as a good idea for them." Took me a bit to learn this in engineering, you would think "but it will be correct!" would be an unassailable argument but as the author notes, the person who has to sign off on it may not care if it is correct, even if you passionately do care. This works for everything at the job, not just fixing software correctly.
If you have to explain why quality matters, they're at least as ignorant as you if not more. They deserve their fate. The upshot is you'll probably also get paid way better and more quickly develop a better sense of how business is supposed to be done at a mature company.
Of course you also have to deliver on your promises that all the time you're spending will improve things and isn't just some amateur quixotic itch.
We can roughly say we can have three out of: (high) quality, (low) time, (low) communication complexity, and (low) money. (time is a dependent here.)
People are trying to apply factory processes and structures to a team sport, an engineering discipline. You do not teach or build a basketball team by breaking down each attack phase into steps and checkmarks.
You try to minimize communication and make the team work as one. It is a team and individual building, not a process building exercise. You make a plan, and follow the Moltke's the Elder conclusion:
"no plan of operations extends with any certainty beyond the first contact with the main hostile force."
(Or paraphrased as you have heard: No plan survives contact with the enemy.)
All (types of) Engineers know this. But software engineering is "special." And it is not a "move fast and break things issue." That is part of all engineering or team playing too.
It is the type of business mentality, that because a plan did not go exactly as expected we need to add more process. Whatever that process may be. Because if "I as a manager add a process, then the next failed plan, I am covered, and I will blame the individuals."
Process has a place to ensure things happen in a legal and moral framework. And minimize adverse circumstances -- e.g. we bet all the hedge fund money accidentally when running tests.
Process is used differently in most startups and corporations with not the team in mind.
Do you ever yell at a traditional architect and ask them when it's going to be done? It's always when the client is happy or makes their mind up about it. A lot of dev is like this.
Every industrial practice.
On the other hand, the title just means that programming is not an industrial practice. What should be obvious to anybody that looked, but some people insist on not seeing it.
For engineers, the situation is very similar to software. There are many tools for enforcing quality, but there's no structure for teaching the engineers, and no, there isn't a widely accepted theory for how to teach design quality either.
The one place where people are consistently taught how to build quality is on manufacturing.
Imagine if every software company was obliged by law to:
- Every single release has to have been signed off by someone who got their "software release engineer" certification at the software equivalent of the FAA.
- This engineer is required by law to not sign off unless every box on a 534 item checklist has been manually verified.
- Any time an unplanned downtime happens at any company, a government team comes in to investigate the root cause and add points nr 535 through 567 to the checklist to make sure it never happens again.
If such a system was mandated for software companies, most of the common bugs would very rapidly become a thing of the past. Development velocity would also fall through the floor though, and most startups would probably die overnight. Only companies that could support the overhead of such a heavyweight process would be viable, and the barrier to entry would massively increase.
It's specific to the complex and unsafe C++ language though, rather than addressing broader software development methodology.
The rebuttal to your implied Goodhart's Law <https://en.wikipedia.org/wiki/Goodhart%27s_law> that was offered by my manager was "tension metrics" <https://en.wikiversity.org/wiki/IT_Service_Management/Contin...>
If I understand his theory correctly, in your case there would be a competing metric to the "test coverage" one that said for any changeset, a test cannot itself change by more than 20% in the same changeset as non-test code. So you can change the code such that it still passes the existing tests, or you can change the test to adapt to new requirements, but you cannot rewrite the tests to match your newly changed code
I'm acutely aware this is a terrible example, the devil's in the details, and (in my experience) each company's metrics are designed to drive down their own organizational risk <https://en.wikipedia.org/wiki/Conway%27s_law>, combined with "you're always fighting the last war" :-D
The entire thing is terrible from principle. You won't find a good example, because that's not how you use a process checklist.
That's the reason why aviation can only shine when it becomes a private means of transportation, and I don't mean 70mm private jets but, 150k light helicopters.
When a critical mass is hit then accidents will become no more traumatic to the collective psyche than car accidents, the lighter the aircraft the better because it would seem exactly like a car crash as opposed to leaving a huge burning hole into the ground
Machine handling requires a lot of training and a lot of well thought out procedures.
I cannot fly professionally anymore due to health, but this is something we are taught in aviation and something I too find lacking from tech so far.
Like, you’re taught the standards as part of learning to fly, but as time goes on, you’re told to narrow your tolerance of what is acceptable. So if you are learning how to do steep turns, for instance, the altitude standard is +- 100’. You’re taught, “that’s the minimum, you should be trying for 50’” and then 20’, then the absolute best performance would be where you do your turn, the needle doesn’t move, and as you roll out on a calm day you fly through your wake. But the goal is “better, always better, what can I do better?” And flying is not graded on the overall performance, if you don’t do satisfactory everywhere you fail. Culturally satisfactory, isn’t, it’s the starting point.
That encourages a much more collaborative model I feel like. I’ve only worked one or two flying jobs that were not collaborative. In the outside world it sometimes feels the opposite. In flying, you truly want everyone to succeed and do well, and the company does. Even the guys I hated that I flew with, I didn’t want them to fail. If they failed, I was partially responsible for that.
It wasn’t always perfect, and I worked with some legendary assholes while I was flying, but truly, they supported me and I supported them, and if I screwed up (or they screwed up) the culture required that we found a way to minimize the future potential screwups.
You’re actually trained on what quality means in a wide variety of contexts too, and flight operations quality assurance (FOQA) is a big part of many airlines. In smaller bush operations where I worked, it is significantly more informal, but we truly had a “no fault” culture in nearly all the places I worked. It’s not perfect, but that’s the point, “ok how can we make this better?”
If someone had an idea for how to do something better, there may have been friction, but that was rare if it actually was better, and as soon as you could show how adoption was fast even at the less standardized operations I worked at.
Not saying it’s all unicorns and rainbows, but I feel like quality, and decision making, and “doing the right thing” were an integral part of the culture of aviation. “The weather is too bad and I cannot do this safely” grounds the flight, you don’t blast off into the shit (at reputable operators) to get the job done anymore (it’s not 1995), and it feels like this new industry is the opposite.
The entire concept of a “minimum viable product” is somewhat illustrative of the problem. It shouldn’t be the “minimum viable” it should be the “minimum quality product we’re willing to accept as a starting point.” But that doesn’t roll off the tongue in the same way.
We shouldn’t be striving for anything that’s the “minimum.” The minimum is only the beginning.
Is that not the case in software? The incentive to improve may not be quite as strong as in aviation (crashing software isn't quite the same as crashing airplanes), but it is still pretty strong. Life is very miserable in software when quality isn't present.
Of course, management should be supportive of quality improvements, but their reality is either one where they are under genuine pressure to deliver projects X and Y to stage of quality through to not understanding or caring about quality.
My own experience is that individual programmers have vastly different ideas of quality is based on their experience and education. You can be struggling to get a team to improve and then you hire a somewhat normal individual with a very different background who makes a sizeable impact on quality and the culture of this in the team. I'm thinking specifically of someone who joined from aerospace, but I've seen it with finance backgrounds. I think the background matters less than the perspective and ability to hold people accountable (including yourself.)
No doubt the same as when the members of your garage band are happy to stay in the garage while you have your sights set on the main stage. You either suck it up and live in the misery, or you get better on your own time and leverage those improvements in the quality of your performance to move into a better position where quality is valued.
Imagine I have a deadline of Jan 15th to demo features x, y, and z to senior leadership (who then will make funding decisions that could impact my project), and I get to Jan 15th with x, y, and not z - or worse, none of them working fully. But the code quality is high, there are no TODOs hanging around, no extra development focused logging, no duplication of code, no leaky abstractions.
That is a 100% fail in leadership's eyes, unless you have a really good story on why z didn't get shipped (and code quality is NOT that).
All those things I listed will have to be addressed at some point, and the longer they go without being addressed the harder they will be to address. But if you need to do that to meet deadlines, then you do it.
Of course, if you are in a place where leadership allows you to work from a backlog and demonstrations and features to demo are scheduled not arbitrarily based on leadership's schedule/interest, but on the features that are newly shipped since the last demo, then you are in luck.
At the end of the day the important thing to remember is that you are not being paid to build software. You are being paid to provide a solution to your customer's problem. Other than CTOs and some forward thinking leaders, they don't care about the software. They care about whether the problem is solved, did it cost me more or less than expected in labor and materiel, and is it compliant with necessary laws/regulations.
Though I think that if leadership are not consulting the Engineers on timelines, instead just dictating the timelines to them, then there is a massive problem afoot.
or maybe, them who know how to do this are just unable to spread this knowledge... something about how they think their private secret codes are the source of their wealth
when in fact, it's merely the scheme by which they mantain an advantageous capacity to extract energy from them seeking to learn how to build quality software
There's an obvious comprehensibility complexity to code to anyone who has spent almost any time what so ever trying to make something happen in software. However, we've got zero academics or theory around it.
Just 'best practices' (ie a thing other people are known to do so if things go wrong we can deflect blame).
And code smells (ie the code makes your tummy feel bad. yay objective measures).
And dogma (ie "only ONE return point per function" or "TDD or criminal neglect charges").
Sure, please do something for QA because it'll be better than nothing. But we're probably a few decades of waiting for actual theoretical underpinnings that will actually allow us to make objective tradeoffs and measurements.
(One exit point is a pet peeve of mine since it often makes the code a lot harder to read and think about vs exit asap)
There are pretty much no languages left today where it's even possible to violate this principle without really trying, it's not about having single a return it's about all the functions starting at the top and return statements always taking you back to the same place in the code.
Usually available as optional lectures during the degree, or later as Msc and PhD subjects.
Although, so far I've only bumped into cyclomatic complexity (with some studies showing that it has worse predicting power than lines of code) and lines of code.
Even though I find cyclomatic complexity uncompelling, it at the very least can slurp up code and return a value. Nicely objective, just not particularly useful or insightful to whether or not things are easy to understand.
The provided link looks suspiciously like they're going to talk about the difference between system, integration, and unit tests. The importance of bug trackers. And linters / theorem provers maybe.
I don't think these are bad things, but it's kind of a statistical approach to software quality. The software is bad because the bug chart looks bad. Okay, maybe, but maybe you just have really inexperienced people working on the project. Technically, the business doesn't need to know the difference, but I would like to.
"Measuring Complexity of Object Oriented Programs"
https://link.springer.com/chapter/10.1007/978-3-540-69848-7_...
I don't suppose you know where I can get their list of references without hitting a paywall? Specifically [16] and [24].
EDIT: [For anyone following along]
The linked paper is Measuring Complexity of Object Oriented Programs. Although, the paper isn't free. They reference several other papers which they assert talk about OO complexity metrics as well as procedural cognitive complexity, but unfortunately the references aren't included in the preview.
Apparently, there's also a list of Weyuker's 9 Properties which look easier to find information on. But these look like meta properties about what properties a complexity measurement system would need to have [interesting, but they don't really seem to comment on whether or not such measurement is even possible].
It looks like a lot of this research is coming out of Turkey, and has been maybe floating around since the early 2000s.
EDIT EDIT: References are included at the bottom of the preview.
EDIT EDIT EDIT: Kind of interesting, but I'm not sure this is going to yield anything different than cyclomatic complexity. Like, is this still an area of active research or did it all go by the wayside back in the early 2000s when it showed up? The fact that all the papers are showing up from Turkey makes me concerned it was a momentary fad and the reason it didn't spread to other countries was because it doesn't accomplish anything. Although, I suppose it could be a best kept secret of Turkey.
Renamed programs are defined to have identical complexity, which is pretty intuitively untrue, so I've got my concerns.
EDIT ^ 4: Doesn't seem to be able to take data complexity into account. So if you're dividing by input, some inputs are going to cause division by zero, etc. You might be able to jury rig it to handle the complexity of exceptions, but it looks like it can mostly handle static code. I'm not sure if it's really going to handle dynamically calling code that throws very well. I also don't think it handles complexity from mutable shared references.
Nice try, but unless there's a bunch of compelling research that no actually this is useful, I'm not sure this is going to cut it. And at the moment the only research I'm finding is more or less just defining functions that qualify as a cognitive measure under the Weyuker principles. I'm not seeing anyone even pointing it at existing code to see if it matches intuition or experience. Happy to be found wrong here, though.
The main point still stays, "You are never taught how to build quality software" is wrong, and plenty of Engineering degrees do teach about it.
I was thinking there should be some metric around d(code)/dt . That is, as the software is used, 'bad' code will tend to change a lot but add no functionality. 'Good' code will change little even when it's used mode.
An example of such research done well can be found in Exploring Science, by Klahr. The author and his colleagues look very closely at how people interact with a system and experiment with it, leading to wonderful insights about testing processes. I've incorporated those lessons into my classes on software testing, for instance.
The Power of 10: Rules for Developing Safety-Critical Code[1]
Assessing the Relationship between Software Assertions and Code Quality: An Empirical Investigation[2]
Cyclomatic Complexity and Basis Path Testing Study[3]
[1]: https://web.eecs.umich.edu/~imarkov/10rules.pdf
[2]: https://www.microsoft.com/en-us/research/wp-content/uploads/...
[3]: https://ntrs.nasa.gov/api/citations/20205011566/downloads/20...
It's worse than that. No one can agree what "quality" means.
Mostly, the word is used as a weapon.
The pointy end of the weapon is what management pokes you with whenever anything unexpected happens. Typically they do this instead of making sure that problems do not happen (a.k.a. "management").
The weapon's handle is sometimes flourished by process gatekeepers who insist on slowing everything down and asserting their self-worth. This is not good for throughput, anyone else's mood, or eventually even for the gatekeepers.
People usually refuse to talk about quality in terms of actual business metrics because if anything unexpected happens that's not covered by the metrics, there will be fault-finding. And the fingers pointed for wrong metrics are typically pointed at middle management.
Like in all things here is how you do it:
1. Build a first version that works and accomplishes all initial business requirements.
2. Build test automation or inspection criteria.
3. Reflect upon what you built and watch how it’s actually used in production.
4. Measure it. Measure performance. Count user steps. Count development steps. Determine real cost of ownership.
5. Take stuff out without breaking business requirements.
6. Repeat steps 2-6.
That is how you build quality whether it’s software, manufacturing, labor, management, whatever.
In my own software I devised a new model to solve for this that I call Single Source of Truth. It’s like DRY but hyper aggressive and based upon empathy instead of micro-artifacts.
once again bringing up the hot debate of "are colleges job prep or research institutions"? Many students these days will proceed to grab a job, but is that something a university should strive to be?
I wish at the bare minimum there were more proper apprenticeships for companies that want some specific type of software engineer instead of 3 month vibe checks as it is right now. Or bootcamps made more or less to study to the test instead of what actually brings value. But I guess no one is really chomping at the bit to change the status quo
Universities taking public money (including students government grants/loans) should strive to make society better. Part of that is getting kids into good jobs that society needs done.
There are a few "retired" people taking classes that they are paying for on their own just for fun. If those people think they are getting value despite taking subjects society doesn't value I'm fine with that. The majority of students though are young people that society is trying to invest in to make a better future, and so the more universities prepare those kids to make a better future the better.
Just because companies decided to start requiring degrees because they're too stuff to pay their own staff, IMO they shouldn't get to divert universities from their original mission, which is education and research. Afterall, those are also important to society, and if they don't do it, who will?
For some classes of bugs it can (e.g. if the software is so bad as to open you to a class-action lawsuit; in b2b software bugs that put you in breech of contract), but for many classes of consumer software, it's not clear to me that shipping software that works better is rewarded. Picking not-too-buggy software ahead of time is hard, people are slow to switch (even when the people encountering the bugs are the people selecting the software, which is often not the case), and people are good at subconsciously avoiding triggering bugs.
But at that scale (in my experience), QA is up front and center and is typically a core pillar of engineering orgs.
From a cynical point of view, if those hundreds of thousands of users will use your product despite the bugs, does it matter?
I'm super cognizant that the cited example of "social media company" is its own special ball of wax, due to the network effect, and I wish to holy hell I knew how to fix that bug :-)
Pity that he didn't go to a proper uni. Writing quality software is a solved problem, just not applied in most companies.
No shame to the author for their choice; replacing stock images with generated ones is a great use case. It's spooky to me that we've been so quickly trained to identify this subconsciously.
Algorithms are hard, and take serious work, they're basically math. University seems like the idea place to learn A* search or quicksort, I can't imagine figuring that out without a lot of effort and time spent just focusing on that.
What I don't understand is why programmers themselves focus on algorithms over stuff like "Vague heuristics to guess if this JS framework you tear your hair out in a month".
That kind of this isn't really hard and doesn't require any heavy discipline and focus, it's just something you pick up when you've tried a few different systems and notice stuff like "Oh look, that linter caught this bug for me" and "Hey look, there's not much autoformat support for Mako, maybe I'll just use Jinja2 like everyone else".
Instead they tell us to do stuff like code katas and endlessly polish our coding and algorithm skills rather than practice working on big stuff and managing complexity.
Maybe it's because algorithms are the hard part, and being really good at that is needed to work on the real cutting edge stuff.
Here's my take on testing[0].
I write Quality software. It's a matter of personal satisfaction. Many folks can't really tell the difference between relatively good software, and very high Quality software, but the cost difference can be quite large. We don't get rich, writing top-shelf software.
The issue is, in my opinion, that we don't even produce much "relatively good" software, these days.
I won't get into it. It doesn't win me any friends. I was fortunate, to find a company that shared my values. Many of their processes would absolutely horrify a majority of folks on this site, and it was, quite frankly, often maddening.
But you can't argue with results. They produced extremely expensive kit, for over 100 years, and they are regularly used on the ISS.
[0] https://littlegreenviper.com/miscellany/testing-harness-vs-u...
It’s not everyday that devs like me get to learn from someone with as much experience as you have. Thanks again for sharing your knowledge!
But I am quite aware that I have a ton more to learn, and places like this, are a good place to do that.
I should get around to doing some more writing, though. I've been very involved in a project, for the last couple of years, and haven't taken the time to write.
Now that the project is approaching release, I may be able to free up some time.
Software engineering quality isn't something you teach, it's something you learn.
I've worked on so many poor-quality projects. I've had so many heated conversations trying to teach people that there are no such things as rules in SE and it's important to _know when to bend/break the rules_. Rigorously following rules doesn't mean high quality, automatically. Some things simply don't exist in the little world you're building.
Translations, for example, usually live outside of any layered cake you are making. If you write a string that will be shown to a user deep down in your code, it needs to be marked for translation and translated. If you try to layer it ... gods help anyone who comes along to change that switch statement.
There are lots of other cross-cutting concerns that people try to put into a box because of "rules" that don't belong in a box. That's usually where your quality starts to plummet.
The most useful pieces of code typically have to deal with some of the most ridiculous constraints. Determining a way to integrate these such that the UX doesn't suffer is the real talent.
The only pedagogical model that makes sense for me anymore is the apprenticeship. You can definitely learn it on your own, but it's a hell of a lot faster to observe someone with 20k+ hours XP and skip some of the initial lava pits. Perhaps some of those lessons are important, but learning and knowledge are not linear things.
And the good thing about snapshots is that they provide a visual way to communicate a preview to stakeholders of what may change, both for the currently-being-developed workflow as well as to prevent visual regressions elsewhere - so they're inherently more easily justifiable than "we're spending time on testing that you'll never see."
The article is correct that treating QA as the last phase of the project, and a disposable phase at that, is a recipe for disaster. But if you make it an ongoing part of a project as individual stories are burned down, and rebrand it as part of a modern development workflow, then it's an entirely different conversation.
This is a big area that I wish researchers would focus on. What really does make for long term high quality software. What are the actual trade offs. How do we mitigate them. What are the limits to TDD - is it good? Detroit or London mocks - when to use each? When is formal verification worth the time and effort? There is a lot places where we as an industry have ideas but are in heated debates without knowing how to find a truth.
It certainly is not something with a single easy answer, and I certainly agree that it remains a fruitful thing to continue researching, but that doesn't mean that there is nothing to be taught about it. There is lots to be taught about this, and lots that is taught about it, to lots of people.
I agree with you on that one. IMHO it's because of that difficulty to know when to optimize for quality, but also because of sheer incompetence.
It's akin to pollution, really: Individual company making 'dirty' things won't see the consequences. But scale this mindset out and suddenly we wake up in a world when trillions of dollars are spend to counteract those effects.
I wonder where you get the confidence to make such a strong statement, which is clearly not warranted. I want to challenge you to broaden your view a bit: Not a lot of software is like Windows explorer. Not a lot of software is performance critical. A lot of software can do with bugs, with many many bugs, to be honest. A lot of code is run fewer than 100 times, before it's retired. Also, not a lot of software written has many users. Or enough users to support maintaining it. "Pollution" often affects just the author of the software themself. Software is just a means to an end, and the end decides, how much effort was warranted in the first place.
Not being performance critical doesn't mean it is justified to diss-respect users by wasting their time.
>A lot of code is run fewer than 100 times, before it's retired. Also, not a lot of software written has many users.
Obviously we aren't talking about some simple automation scripts here.
>"Pollution" often affects just the author of the software themself.
You are misunderstanding the pollution analogy. I'm not talking about polluting the codebase with code smell.
I am talking about costs of low quality being non obvious and only revealing themselves at global scale
This is moving the goalpost, and also ignores the fact that software exists on a spectrum from "simple automation script" to "messaging app used by millions". It seems you have a very narrow view of what software is, or what it is used for, and the constrains that apply when building it.
Running that kind of software for the central bank here. So kind of disputing your statement.
> So I am simply excluding this category as an extremum.
Which ignores the long tail. Great approach.
>Great aproach
Unironically better, then trying to make prescriptions as broad and general as possible, because those usually are too generic to carry any actual value
Also note that I wrote "code" originally, because there can be programs which are run very often, but certain code paths are not, so my statement applies even for some parts of popular software.
The image I think would be valuable for you to consider is a curve, where 20% of code has 80% of all executions, and 80% of code get's the rest. It makes sense to put in a lot of effort into writing the top 20%, but on any given day it is very likely you'll be working on the lower 80%.
I have written non-trivial systems deployed in 20 different sites that have never had a bug ever.
My best are usually second systems, focus on simplicity, standardisation and resist scope creep.
100% coverage with unit tests. 100% coverage with integration tests.
I've written a many things with zero bugs after delivery.
(and other that were never ending quality nightmares)
If I am under pressure to deliver I get strict with TDD, since no time for bug fixing.
Where I work is 50% qa effort at least, 50% by budgets, and devs do much >50% automated testing as part of what we call development.
Anything less and projects take longer, because bugs found later cost so much more to fix.
Cem developed a great testing class for Florida Tech, later commercialized under the name BBST, that drew in contributions from prominent industry voices. Otherwise, I don't know of any other University that has put the effort in. An exception may be what Tanja Vos is doing at Open University.
The problem I have with CMU is that they are the source of the Capability Maturity Model, which is simply incompetent work, justified by no science whatsoever, which tries to apply Taylorist management theory to software engineering. It never caught on despite all the marketing they've done-- it's only popular in military circles. I shudder to think what they are telling students about testing.
You can disagree with me, of course-- but in doing so you have to admit there is at least a schism in the industry among various schools of thought about what testing is and should be. My part of the field considers testing to belong to social science, not computer science.
Just an anecdote, but, we had a "Software Development" class like this in CS (I took it in the '90s) and even though it followed a waterfall development model[0] and we used Gantt charts, QA (testing) was a big part of it and 1 of our 4 team members (or maybe 2 of 4 worked on it together) was primarily responsible for it. (I wrote the parser and the made diagrams/documentation for the parser.)
The description (in an old catalog[1]) is:
Software specification, design, testing, maintenance, documentation; informal proof methods; team implementation of a large project.
Turns out I didn't need to look up the old catalog because the description is exactly the same still! Except it's CS 371 now, and the longer "Learning Outcomes" for the course has some newer stuff (agile and version control) but otherwise is all the same things I learned at the time.
[0] https://en.wikipedia.org/wiki/Waterfall_model
[1] https://nmsu.contentdm.oclc.org/digital/collection/catalogs/... (C S 372 in the lower right corner)
For example, the largest technical UAS in Austria offers Computer Science as an undergrad degree but only does research on Software Engineering at a graduate level.
Even for the QA advice in the post "writing tests as you write the software" one can argue it's infeasible/inappropriate for my type of project and/or with the people who work on this project.
So my two cents on this is let Uni teach the students *to be aware of* all the SE principles and best practices this industry now have, like the tools in your toolbox, also let us know that whether to use them in real-life projects, need assessment of situation, cost/value balance, people etc, afterall not everybody works on dream projects.
IME, “QA” doesn’t really correlate with quality software, nor is there really a time vs. quality trade off. Bad software is often also delivered poorly, and high quality software can be delivered quickly.
How cartoonishly incompetent have we allowed managers to get that they can't connect stability to money on their own? If the stability concerned a bridge, would engineers also be expected to translate a potential collapse into monetary terms to get their manager to approve a higher grade of steel cable?
You can mix those three together and produce a desired output, but don't expect perfection, perfect software only appears when the three variables tend to infinity
I suspect that a lot of the bad code that is out there exists because teams are constantly in crunch time where it is important to get certain features out by a deadline.
From that perspective, this statement is kind of a contradiction. If every minute is vital to finishing a feature on time, then the act of writing tests will always block progress of the project
Basically, I just wish it were more innately profitable to write quality software
So you will deliver that feature, it will fail for your customers, and you will be in panic-mode trying to fix it on live systems. Seriously, that's going to happen.
Finishing a feature must always include at least some minimal time for testing. That testing needs to be done by someone not involved in the development. Developers sometimes misunderstand requirements (or think they "know better") and they will test what they implemented, not what the requirements actually say.
HOWEVER, the real culprit are the MANAGERS.
After 50+ years of skanky software development policy aimed at low balling cost its time to blame the right people. No amount of cajoling, "business/budget" speak manipulation is going to fix a fundamental flaw in how managers at that level are trained and behave.
We have stop being apologists for mistakes not of our making. If using QA techniques would increase the managers bonus then we will see it being used. If all it does is make better software then this post will be rewritten in 50 years and would still be relevant.
In my experience it is generally wise to avoid abstractions and copy/paste things a couple of times, once the code base matures good abstractions will be more obvious. Even then it's good to think about future changes, will these 2 things want to evolve separately in the future? If the answer is YES, then maybe coupling them is not a great idea. I think there was a really good Kent Beck talk about coupling vs cohesion somewhere.
Another thing to think about is breaking things, if changes are local to one single endpoint then any changes there can only break that endpoint, edge cases and scenarios to consider are only relevant to that endpoint. When changes to a core abstraction are required then hundreds of use cases/edge cases need to be considered - why are we creating so many core abstractions in our systems in the name of DRY?
I've also found that the more moving parts you add the harder a system becomes to learn, the S in SOLID is probably to blame for that. The only single responsibility principle is useful for is unit tests (easier to mock), but many times harder to understand. If the actual functionality is not local to the file things become ungreppable via code search, understanding the entire system requires an IDE and jumping around to each and every beautiful lpad() implementation and trying to piece what is happening one 3 line function at a time.
Then there is also layering to consider, if 2 pieces of code look somewhat similar but belong to different layers (example controller and DAO layer, then also care must be taken to not make an abstraction that couples these together, or to couple 2 unrelated modules together that could otherwise have their own life cycle).
These are just some aspects I could think of that I think about when creating abstractions, but somehow I see engineers focus too much on DRY. Maybe they got burned so bad some time in the career by forgetting to change something in 2 places?
The buyer has a specific requirement, which is distilled into a specification. The specification is implemented. Implementation that doesn't match with the specification is a bug.
Now in order to verify the implementation's correctness relative to the specification there must be a QA.
The idea that people forgets to add QA in the software development process is wild because it means people are forgetting how to conduct business.
So I'm gonna need to see dome data here. How many universities has this guy surveyed to make a claim like that?
My university did teach me design patterns, architecture and all. My professors did give me projects where requirements would change mid way and the software had to adapt without being completely rewritten. They did teache to write unit tests, to use a build tool to generate docs, reports, etc.
Was I battle tested for my first job? No, but it really wasn't as bad as what's described here.
"I want to start programming, what language should I learn?"
"One of these two most popular languages, where you will not even know about basic errors until you run the code!"
Binary arithmetic doesn't make dollars for your bosses, but its fun. That doesn't make it less significant.
I highly recommend anyone wanting to learn how to do something with care, try to pick up a skill that requires you to do it right the first time.
You need to understand the domain, you have to design something that solves an actual problem or delivers tangible value to someone, you need a holistic approach to user experience.
I've seen this too many times to count.
If it's slim to non-existent, you'll be working more hours with more on-call "incidents" and no life.
For example, at 16, my QA manager (whose husband had a Nobel Prize) and my tech lead (who wrote malloc/free for Unix and was learning VMS from me as fast as I could learn C from him) were totally opposite forces, separated by HR and different directors. QA didn’t code anything but batch jobs/shell scripts and contracted back to Engineering for any apps they wanted (or simply required them to be in the release package). If you wanted to keep your job, you didn’t submit programs with performance/functional deficiencies and definitely not logic issues as compared to the SPEC that all parties signed at the outset of the project (put that in yoh Kan Ban, Man rofl)
Ok another arrogant post from older former child prodigy! Happy Holidays!
Some of it can't be taught though, any more than a craftsman or artist can teach a novice how to produce what they do with the same result.
The processes and steps can be taught, but only through experience can some things be internalized.
Kind of related to https://english.stackexchange.com/questions/226886/origin-of...
What you read on the internet is mostly ... bullshits.
What are your quality gates? How many open bugs are there on production? How often you close bugs with resolution: won't fix(because of budget issues)? How often do you have production incidents? What is your testing strategy? How are you testing your requirements before assigning them to the developer for implementation? Do you hire external professionals for security testing?
- why is testable software higher quality? Does it add value to the software? I'd venture that untestable software has the same value (if not more) than testable software (due to time-to-market). You can write software that is 'obviously correct' and "high quality" at the same time, without any tests.
- Why does software that can adapt to new features increase the quality? If that is the case, we must argue that WordPress is extremely high-quality software. Or SAP.
- How does architecture influence quality? If that is the case, then there isn't any need for different architectural styles since there should be "one true style" that has the best quality software.
Sacrificing quality assurance to favour other aspects is common, but the quality usually suffers.
A company favouring time to market over testability is likely to release buggy software. They can get away with it.
Adaptability is a common quality, but you can find counterexamples. WordPress and SAP are successful software that may not check all the quality boxes.
Some architectures are for sure worse than others, and there isn’t one good architecture for all kinds of problems.
> why is testable software higher quality? Does it add value to the software? I'd venture that untestable software has the same value (if not more) than testable software (due to time-to-market). You can write software that is 'obviously correct' and "high quality" at the same time, without any tests.
Note I said testable software, not software with tests (there is a difference!)...I'd agree that software with tests (which is by definition testable software) has a huge developer cost to it that may not always be in the best interest of the company (like you said, time to market might be important). But in my experience, writing code in a way that can be tested later is only marginally more costly (time-wise) than writing code that isn't. A good example of this is writing modules that communicate with message passing and have state machines over direct function calls. The former has a slightly higher cost for dev time, but you can always retro-fit tests to it once you've achieved market penetration. You can't always do that with direct function calls.
> Why does software that can adapt to new features increase the quality? If that is the case, we must argue that WordPress is extremely high-quality software. Or SAP.
This is a good point that you bring up. I think what we are getting at ultimately is that quality and value are distinct entities. Software can have high value without being high quality. In my mind, being able to provide the business with new value-producing functionality without causing a spike in bug reports is my (admittedly vague) standard.
> How does architecture influence quality? If that is the case, then there isn't any need for different architectural styles since there should be "one true style" that has the best quality software.
Architecture has to match how the software teams communicate with each other. Like actually communicate, not how the org chart is made (see Conway's Law). So my point is then that if there are two separate teams, your code should communicate between two "modules" that have an interface between them. Just like real life. It would be silly to implement a micro service architecture here. That's why Amazon's SOA design works for them: it matches how teams are organized.
- Who gets to define testability?
- I want to add a coffee maker to my crash test dummy; is the lack of room for the filter and water tank a sign of a bad design? Or not flexible enough for my feature?
- (cue meme) "You guys have organizational structure?"
- Who gets to claim the impedence mismatch? What are those consequences? Wait, where are the dependencies defined again outside of the software?
I do (just kidding!)...Testability is the ability to add testing at a later point. There is no hard definition of this, but if you can't test at least 75% of your public facing functions then I'd say you don't have testability. Remember testability means you can have a tigher feedback loop which means that you don't have to test in production or in the physical world. This means you get where you want to go faster.
> - I want to add a coffee maker to my crash test dummy; is the lack of room for the filter and water tank a sign of a bad design? Or not flexible enough for my feature?
I know you are joking, but imagine for a second that your business did in fact invent a brand new way to test crashes and that coffee makers were the key to breaking into that market. If the dummy can't accommodate that then...yes! It is a bad design, even if it was previously a good design.
> - (cue meme) "You guys have organizational structure?"
Remember: there always is an organizational structure, with or without a formal hierarchy. You want to match your software to the real one.
> - Who gets to claim the impedence mismatch? What are those consequences? Wait, where are the dependencies defined again outside of the software?
There are no "the company blew up" consequences with this type of failure mode. Instead you get a lot of "knock on" effects: high turnover, developer frustration, long time to complete basic features and high bug re-introduction rates. This is because software is inherently a human endeavor: you need to match how it is written to how requirements and features are communicated.
It’s not that crazy.
If this is given, every other metric, like features, bugs, performance is just a linear dependence on development resources (maybe except documentation, but that is kinda an externality).
But how do you define the requirements is the real question (and problem)…
Plain imperative structured code is usually the cleanest and easiest to understand. Introducing abstraction too early I think confuses people and encourages complexity where it's not needed.
Yep!!
References?
It sounds great - i am sure this has been studied. Anyone have any studies that have been done?
When you truly understand the software you are writing then, and only then, can you communicate it logically in code for the computer to execute and, much more importantly, code for a person to read. Well written and well understood code means it’s very obvious what you are doing. Later, when the code has a bug or needs to be rewritten, then it will at least be clear what you were trying to do so that it can be fixed or extended in some way.
So then the question is how do we train people to have these skills? In school, science experiments are a good way to teach logical reasoning and communication — here is what I thought would happen, here is what I did, here is what happened, and here is what it means. Math teaches you how to reason abstractly and, again, prove your point with logic. It’s a slightly different beast in that it’s harder for a math experiment to go wrong. It’s also harder to come up with and overcome novel scenarios in the lab with math, so in all it complements science well. And of course reading and writing English build your ability to express your thoughts with words and sentences. Many other high school subjects combine these in various measures — history for example is data gathering, fuzzy logical deduction, and reasoning in written language.
The bottom line is that quality software starts by working with well educated people and conversely all the most abhorrent heaps of over coupled illegible nonsense I’ve seen has come from people who, to be blunt, just ain’t that smart or well rounded, intellectually.
It’s a principle I carry over to hiring: smart and well educated wins out over pure-smarts.
Whoooosh
The two overlap most of the time in subtle ways where the science gives an important foundation, such as learning Big O notation and low level memory concepts where exposure helps. I've personally seen this with a smart coworker who didn't go through university and is great at programming but I'll catch him on certain topics such as when he didn't know what sets and maps were and when he tries to sleep a second instead of properly wait on an event.
However, the differences between computer science and building software are problematic. Watching my wife go through university, she's had to struggle with insanely hard tasks that will not help her at all with software, such as learning Assembly and building circuits. The latest example is the class where she's learning functional programming is not actually teaching it to her. Instead, they combined it with how to build a programming language, and so instead of giving her toy problems to teach the language she is having to take complex code she doesn't understand well that generates an entirely different programming language and do things like change the associativity of the generated language. In the end, she feels like she's learned nothing in that class, despite it being her first experience with functional programming.
On the flip side are the things that are necessary for software that aren't taught in university, like QA. For me personally, back when I was in university a decade ago I never learned about version control and thought it was just for back up. Similarly, I never learned databases or web, as the required classes were instead focused on low level concepts as Assembly and hardware. My wife is at least learning these things, but even then they often seem taught badly. For example, when they tried to teach her QA, instead of hardcoded unit tests, they made her give random inputs and check to make sure the output was correct. Of course, checking the output can only be done by rewriting all of your code in the testing files, and if there's a bug in your code it'll just get copied, so that kind of defeats the purpose. Even when the assignments are relevant there is often no teaching on them. For example, her first ever web code was a project where they told her to hook up 6 different technologies that they had not gone over in class, with only the line "I hope you've learned some of these technologies already".