Hacker News Reader

Comment by ryanmarsh | Hacker News Reader

If billions of computers become vulnerable to malicious actors and a few hundred or thousand people suffer dramatic personal damages, well that's a nice and quiet problem which will be quickly forgotten in the 24-hour news cycle.

This. I do development coaching and training. I’m totally burned out because my job doesn’t matter. These companies don’t care. They hire me to “increase code quality” but that ends up being a lie. They just use me to get their teams to crank out shit faster, and I do mean “shit”. They don’t care if the systems blow up a few times a year or once a week. Banks, oil companies, airlines, all of them. They loose millions because of it too, still it doesn’t matter. No one (in leadership) gives a fuck. “Just make this quarter’s goal” is all that matters.

Fucking equifax for crying out loud, NO CONSEQUENCES. If this isn’t ironic enough do some googling on why Equifax is named such. tl;dr This isn’t their first rodeo. They once fucked up so big they had to change their name.

Sucking at software only has consequences in a few industries, usually ones where the product is software, the product is discretionary, and the customer is fickle. Even then it usually sucks.

These same companies are audit conscious (about money), they’re crazy about regulatory compliance, and workplace harassment. Why? These things have real consequences.

The sooner some company gets fined one year’s profit or an exec goes to jail the sooner this shit will actually matter to them.

View on HN

The topic of your rant is something that it took me forever to accept as a software engineer. Nobody cares about quality.

Your method is refactored to be super-fast and have no branches? Nobody cares.

Elegant, re-usable architecture? Nobody cares.

You've optimized an inner loop in assembly and increased speed by 30%? Nobody cares.

Made it to zero compiler warnings? Nobody cares.

No known crashers left in the bug tracker? Nobody cares.

Fixed that bug that's been in the code for two years? Nobody cares.

Patched a security hole that could allow a crazy bad attack? Nobody cares.

The vast majority of software shops out there care only that something--anything--ships as fast as remotely possible, and as long as it does not kill the customer, it's a success. Nobody in leadership gives a single fuck about anything else. There are a few places where this is not true, and good software people tend to collect there, but it's true for the vast majority of places where you will work. The sooner you accept this, the less likely you'll burn out.

If there were a list of such companies, I would patronize them.

My universal advice to graduating CS students is, "no matter what your current code quality/dev speed balance is, your boss will think you're polishing too much."

Second advice. If you want to pretend that quality matters, move to aerospace, finance, or any critical systems.

Third advice. It's not enough to fix bug, everyone MUST know that you fixed that weird bug that customers were complaining about for years and noone could understand the cause.

> If you want to pretend that quality matters, move to aerospace, finance, or any critical systems.

The important word here is pretend. If the system is safety critical you'd think that quality matters. It doesn't. What matters is following the process laid out in ISO 26262 or whatever standard applies to your field. Whether or not that improves quality is not important (it does, but only a little). The important part is that you have reduced/no liability for accidents if you can prove that you did everything by the book.

andai.depression++;

Yep, part of the quality standard for medical devices is to have processes, and also processes to improve and change the processes. So theoretically there shouldn't be any doubt as what to do in a given circumstance when something fails, and that standards would rise as development matures. But in reality it's just a bunch of paperwork, and as far as the software is concerned, standard practices of unit and integration testing, code reviews, and strict compiler settings are more than adequate already.

Edit: It's all about documenting that those things have been done and signed off on, so keeping track of paperwork.

Testing, review, history of changes... many things that are completely unheard of in the average small development shop.

Exactly. A prime example of this is the “quality” software at financial companies.

They care about loosing money and risking to lose money.

I'm on the last few pages of this book:

https://en.wikipedia.org/wiki/What_Do_You_Care_What_Other_Pe...

...where he talks about his role and questions regarding the Challenger shuttle disaster; I don't know if things have changed much since then at NASA, but I suspect they haven't, given Columbia and such.

Mmmyes, but the process implies a heavy chain of verifications. So even when there has been an idiot and a lazy guy, at some later stage in the process, it has to go through someone who cares at some point and it gets fixed at that point.

After all, that is the point of those heavy processes: manage to product something fairly reliable despite the acknowledged presence of morons and shitty managers. The process does not try to eradicate those people, it works around them with some kind of redundancy.

Having been one of those 'concerned' people, I must admit it is exhausting. You know that if some work comes from this guy, or that company, it will be 90% shit and you have to explain them again and again what is wrong and how to fix it, or do it yourself when you are really fed up. So basically you do your job + the slackers' job. That is how the system works. As a system, the result is not too bad. As far as individuals are concerned, however...

And make sure to record it so that at the end of your yearly review cycle, and before the raises are decided, you can remind your upper management of your notable accomplishments.

Either it's easily perceived as bringing value or it won't have any effect.

If you have to remind people of it, it's as good as not existing in the first place.

I disagree. A year is a long time, and a valuable, difficult achievement in February may raise your prestige for a while, but by December that's faded to background levels. I even forget my own achievements from nine months ago, never mind those of a dozen other people on my team.

The question is always:

> What can you do for me or have done for me recently?

Services provided so far in the past that need to be reminded are at best irrelevant and it can actually hurt you to bring them up.

I'm not saying that it is nice, only that humans behave individually and (even more collectively) according to this.

So at my annual review in December I should only be assessed on work in the last quarter? I guess I'd better save my best work for November.

Depending on the attention span of your particular managers (and their managers as well, most likely), this can indeed be a viable strategy.

You might also carefully consider your company's culture when deciding how to split your efforts between new (user-visible) features, preventive maintenance, and bugs affecting end users (especially any VIP end users that have your managers' ears).

For example, a 'Hero' culture rewards last-minute, late night, death march efforts, whether to fix an urgent problem, or to add that one feature that is (supposedly) needed to land a specific high-value customer/sale.

> The vast majority of software shops out there care only that something--anything--ships as fast as remotely possible

When nobody cares about you get the point sooner or later where you can't ship, they've probably launched a product but there's still a lot more to ship in future. Eventually they become so mired in technical debt that they care.

When they realize they've invested a hundred million dollars in something that now has no value? They care. (especially if it's a turd before it gets out the door)

When a start up comes along and eats their lunch because they aren't weighed down be legacy code? They care.

When they lose revenue because they can't keep customers happy? They care.

The problem is that the professionalism of software developers doesn't allow them to learn these lessons early and hard enough.

> They care.

I disagree. They will hire a consulting company to do a root cause analysis. The Ha'vad grads from McKinsey will swoop in, determine that the past doesn't matter, you need to get to black and to get to black, the St. Louis team needs to go, along with 20% of the following 7 teams....

No, they don't care. And never will.

Then they just rewrite it again shittily. lol

> There are a few places where this is not true

Where?

Probably the Apollo moon lander software team, circa 1968.

https://www.fastcompany.com/28121/they-write-right-stuff

This article bothers me because it implies that the path to correct code is simply to have code written by white-collar salarymen (as opposed to unkempt youth), and not improved formal methods.

It doesn't imply there is a correct path to code. It implies that the best path to the goal for this program is to minimize the opportunity for exhaustion to be a variable.

> That’s the culture: the on-board shuttle group produces grown-up software, and the way they do it is by being grown-ups. It may not be sexy, it may not be a coding ego-trip — but it is the future of software. When you’re ready to take the next step — when you have to write perfect software instead of software that’s just good enough — then it’s time to grow up.

but it is the future of software

It's over two decades since that was written. I remember reading it when it was first published. Does it look much like the future?

It might not be the future of software, but it's the software of the future, in the sense that it's what enabled us to get to the moon, and will enable us to get to Mars.

Shops that use strongly-typed functional languages are, in my experience, a good bet. They usually made that choice because they have an unusually compelling need to write correct code. Aerospace also has a few sub-industries where correctness is priority #1 (like commercial autopilot software).

I agree with you, and this is a hard lesson to learn. But I don't think it's necessarily a bad thing.

Software should really only be measured by the value it provides. If a terribly buggy piece of poorly written code still saves hundreds of man hours a week, it's a win. (Unit tests be damned)

If that big ball of mud that's using 20 year old technology still prints a billion dollars a year - that's a win. (Microservices be damned)

If some slapdash jquery-and-duct-tape web app still solves a specific problem I have, that's a win. (React SPAs be damned)

Does crappy software flourish while beautiful software dies? Sometimes. But usually code hygiene, security, bug counts and crashes are not what makes or breaks the success of a piece of software. Developers often think their work is the most important, when in actuality it is usually not. Sometimes you really just are a cog in the machine.

I’m a web developer in my spare time. I write little things that solve problems in my two day jobs, they haven’t tended to be pretty because often I’m learning as I go and when it works then it’s truly time to move on to the next problem, even though I now know enough to solve the old problem again more elegantly. I’m trying to use better practices and modern JS frameworks in new projects just for experience, even though they are actually overkill in many cases. Certainly the people I work for would rather have two useful utilities written in plain es5 and php than one useful utility plus a story about the code quality. I’m gradually finding the middle ground where somebody call look at my code and not instantly want to take a shower.

Saying all this shows that 1. You care and 2. You know the difference.

I’d hire you.

Good to hear- I’m hoping to switch to full time web development some time in the new year. It's actually my favorite thing at these other jobs.

You're forgetting about ethics. Even if your boss doesn't care about security, if you want to be a decent person, you need to at least try to prevent your system from becoming a tool to harm others.

Hanlon’s razor comes through in full force.

The vast majority of people making agregrious security errors know no better. You could argue that whoever hired them shares or shoulders the blame, but to suggest they’re not decent people is a stretch. Most I’m sure are. They’re also likely infosec idiots.

Now if you know something is terribly insecure, understand what’s involved in fixing it, and go out of your way not to, then yes ethics come into play. I see that the same as an engineer (in the true sense of the word) staying silent on an issue involving automobile brakes that could lead to casualties. The consequences are not 1:1 but the ethical question is in the category.

I already have issues when I see something but I know no-one is going to pay to fix it, so what can I do? I report it, no-one cares; I might be able to fix it but then I do this for free.

If your bugs can hurt people (financially, physically) I think you need to work a bit harder to make sure it doesn't sneak into your work. Whatever you believe helps there (unit tests, formal verification, ...); if you didn't do all you could do because you are not paid for it (your boss tells you to add features, don't waste time on things that didn't break yet for instance), what should you do? Seems like a real issue for professional coders as there are not so many options; not many companies will pay to prevent these kinds of things unless they had a big issue before already (and someone(s) got fired / sued for it already).

I'm normally on board with Hanlon, but "infosec idiots" who fail to educate themselves on basic security principles are guilty of negligence at best. There's no reasonable way they don't know that security is a real concern, so they should be looking into it. If they still fail, then we can go back to Hanlon's razor.

But as one HNer to another, I was really talking about your second case. To claim here that security can ever be discounted, as the person I was replying to implied, is pretty much inexcusable. Everyone here knows that security affects more than just your business, so no one here should be solely applying business logic to it.

> Software should really only be measured by the value it provides.

The problem is when the "negative value" is externalized leading to false evaluations. "Oops, so sorry we leaked all your social security numbers. Our bad."

Yes, the "software should really only be measured by the value it provides" attitucde is kind of the key problem we're facing in a whole lot fields.

Because it's not really the value overall the software provides that's being talked about but the instantaneous, immediately visible payoff to a single consumer that software, the people or that consumables are getting judged by. If the software is going to result in trouble over time, if it's going to have security holes that will cost a lot over time, if it's going to commit you to garbage that's updated less and less frequently, etc. None of this calculated. Just as the health costs of sugary drinks don't get calculated, the social costs of poor education don't get calculated, etc.

It's up to regulation - self or government imposed - to factor externalities in, change society and/or change the law

Somehow I never consider to solve a problem by delegating it to government or society. My first thought is always: there must be a technical solution for this. Even government and society are going to be solved by technology: crypto/blockchain or something which will evolve out of it.

There's no tech that specifically gets people to do the right thing. There's tech that might make people do things but that tech can be harnessed to get people to do either right or wrong things.

Also the externalities that are never mentioned: bloat and poor performance unnecessarily takes up user's resources, frustrating them and preventing them from doing other things on their machine simultaneously, and also unnecessarily wasting electricity. Multiply that by the number of users of the software, and suddenly all that talk about "optimizing for developer time" starts to sound like "I'll save $100 for myself by inflicting $100k in externalities".

Even on a larger scale, the world hasn't burnt down with all the private information that's been stolen lately, has it? Have there even been any significant consequences?

Is it possible that the value assigned to keeping that information secret was, in the end, actually appropriate?

It's a waste of the information stolen to use it while people are still on high alert.

"Have there even been any significant consequences?"

Ask someone who's had their identity stolen.

This, ironic enough, is my main motivation to improve my software development skills, from algorithms to system design to development patterns. Because, like first impressions, the first write is the most important. If it's done well the first time around, the true risk of missing an improvement upon the next iteration is significantly lowered. Edit: also the reason I'd like an ongoing education in basic security for app devs

This is what I call the Mediocre Mercenary Mentality, this idea that "as long as we can make money off of it, it's good enough".

This mentality is the reason why sometimes -- but not always! -- my phone won't play music in my car until I restart the _car_.

This mentality is why game development companies get away with charging full price for unfinished, bug-ridden games with crippled features and missing content.

This mentality is the reason why I couldn't see my credit card in the list of my accounts on my credit union's online banking site for several days.

This mentality is why Experian and Equifax don't report the same credit score for me.

This mentality is why I feel ashamed every time a receptionist apologizes for the wait, because they "have been having problems with the system."

Most importantly, this mentality is why I have to worry about whether someone will steal my identity because people keep writing shitty code on top of shitty code and my personal information keeps getting leaked.

When I'm so disgusted by my own profession, no wonder I ended up burned out.

We have an economic system where we reward maximizing short term financial gain. As long as that doesn't change companies will try to sell the crappiest shit they can sell you for the most money.

I think this is more than just the economic system. If you look at people in general, we live in the age of instant gratification.

We want everything fast! Nobody is willing to put in the work and get the reward after a longer period of time... No! We want results now! We don't want to wait for the food to be properly cooked, we want it now, so we go for fast food. We don't want to put in 1 year of work to learn something new so we can feel that we achieved something, we want it now, so we play a game where we can 'win' in 30 minutes. We don't want to build a relationship and eventually end up being intimate, we want it now, so we go for 'one night stands'.

Then you ask yourself... is it weird that the economic system works the same? That the software industry works the same? :/

I don't think the phenomenon you described, as some new thing specific to this age, is in any way real. If anything, the things you mention only reflect the new choices we have. Going over your individual examples:

> We don't want to wait for the food to be properly cooked, we want it now, so we go for fast food.

We go for fast food because it saves time. When I opt to order in instead of going out for lunch, or grab a burger from McDonald's en route, I do that because eating is mostly instrumental. I don't want to eat at that point, I have to, and the sooner I can fill myself up, the sooner I can get back to doing the things I actually care about.

Fast food, and food delivery services, allow people to choose to spend more time on other things, when those other things are more important to them. The same people will enjoy a finely cooked meal on a different occasion, when they prioritize it.

> We don't want to put in 1 year of work to learn something new so we can feel that we achieved something, we want it now, so we play a game where we can 'win' in 30 minutes.

Spending a year on learning something to just get a quick feeling of achievement is a very stupid way to go about it. Videogames are better for that. Learning is better for getting long-term feelings of achievement, and to actually gain knowledge/skills that you can use for something. And again, it's not something new - our generation wastes time on videogames, previous generations wasted time playing soccer, cards, darts, and doing tons of other quick-reward activities.

> We don't want to build a relationship and eventually end up being intimate, we want it now, so we go for 'one night stands'.

One-night stands exist for as long as humans are humans. Nothing fundamental changed, only the hookup methods evolved with population density and available communication tools.

> Then you ask yourself... is it weird that the economic system works the same? That the software industry works the same?

Nah, they don't work the same. In them, the actors are not driven internally, they're driven externally. People write shitty software, or sell shitty products, not because of their need for instant gratification, but because of market pressures. A fast hack sold by your marketing team can be a difference between you getting $100M contract versus not getting it, or getting your product on the market first versus a week after your competitor. Market economy, because of all its efficiency, is what creates the culture of suck.

You say it like is a bad thing. If I compare two service/products/relationships/experiences which are equal in all but one attribute: delivery time, guess which I will choose.. Delivery time is an important attribtue and reduces (subjective) risk of no delivery. And delivery time is an attribute which is measurable. In comparison it is hard to measure how bug free a peace of software is, or how healthy food is, or how successfull your relationship will be, or how good you can learn this new skill in a year.

In b-school Profit=Revenue-Expense was drilled in over and over again.

It's so obvious. But until you realllllly sit down and internalize it... then you can be at peace with the shitty service of Comcast, etc.

In other words: Don't hate the player, hate the game.

> This mentality is the reason why...

The way I see the post you're replying to is either you learn to cope with these things or you'll burn out faster.

Since there's no accountability, if you want to do things the right way, you'll have to do it by yourself and be prepared to swim against the tide..

I have this saying that most software is tent software. Nothing wrong with tents, music festivals are built on tents for example. They are cheap, quick to put up and put down and can do a lot!

You just have realize that most software buildings are tents, and there are a few buildings that you can use out there.

Tents are usually temporary though, software rarely is.

You have been able to put to words the feeling that I have had for such a long time. Thanks.

I don't know if the world is getting a shittier place. But it surely is getting less reliable.

> I don't know if the world is getting a shittier place. But it surely is getting less reliable.

It is, because everything is being optimized into literal minimum viable products. That's how competitive markets work - whatever aspect of your product or business you can cut out to save money and thus get ahead of your competitors, you will cut out, and then your competitors will cut it out too, in order not to get outcompeted by you. This race-to-the-bottom phenomenon is fundamental to how competition works, and the result is that products gradually lose quality.

It's the reason why your grandfather's washing machine probably works 'till this day, while you have to fix your new one every year, and will probably replace it in five years. "Building to last" is a good example of a quality that the market economy optimized out over time across pretty much all products in all sectors.

> It's the reason why your grandfather's washing machine probably works 'till this day, while you have to fix your new one every year, and will probably replace it in five years.

Adjusting the cost of each product with inflation, things really are built like they used to be. Your grandfathers washing machine both cost far more than a current model* (again, adjusted for inflation), and all of the poorly built old washing machines broke and no on sees/thinks about them anymore.

* I am having trouble finding a source on washing machines specifically, however I'm assuming they followed the same trends as most commodities tracked by BLS: https://data.bls.gov/cgi-bin/surveymost?ap

As questionable as citing Reddit is, this post rather well encompasses what I am trying to get at: https://www.reddit.com/r/Showerthoughts/comments/4kn8ku/the_...

> When I'm so disgusted by my own profession, no wonder I ended up burned out.

Ha ha, maybe this is why they don't want older engineers. They're wise, cynical, bitter, skeptical and not likely to suffer bullshit management.

It sounds like you'd burn out in any profession. Perfect is the enemy of the good. If you're holding your work, and that of others', to a perfectionist's standard, then you're just causing yourself unnecessary stress and anxiety.

Most of those bugs and failures you listed are really inconsequential in their contexts.

Car won't play your phone's music? Use the radio. Game has bugs? Play another game until it's patched, or find inventive ways to use the bug. Can't see your CC? You can still charge on it and pay its balance. Experian and Equifax don't report same credit score? Your creditors aren't reporting to every credit bureau.

Ashamed that someone else's system doesn't work? I can't help you there. That's some deep psychological issue.

About 10 years ago I came to understand that mediocre runs the world. All these people, who are your bosses, who are getting raises and promotions, they're the B/C students from college. They don't care about perfect. They really only care about finishing what they're assigned and going on with their outside life.

If you want perfect, do it on your own time.

> Most of those bugs and failures you listed are really inconsequential in their contexts.

Yes, I expect that for most things we could find a context that could render them inconsequential.

> Car won't play your phone's music? Use the radio. Game has bugs? Play another game until it's patched, or find inventive ways to use the bug. Can't see your CC? You can still charge on it and pay its balance. Experian and Equifax don't report same credit score? Your creditors aren't reporting to every credit bureau.

Elevator doesn't work? Use the stairs. Public restroom is filthy? Hold your breath and don't touch anything. Lost a tooth on the right side of your mouth because of an incompetent endodontist? Chew on the left side.

I'm not being facetious here. I really do agree that almost anything is tolerable if we decide to. That last example is from personal experience: I have lost three teeth on the right side of my mouth and I have to chew on the left. Most of the time, I don't even think about it anymore. And it really isn't that big of a deal -- it doesn't have a very significant impact on the quality of my life.

> Ashamed that someone else's system doesn't work? I can't help you there. That's some deep psychological issue.

It could be. Like I commented elsewhere, I would expect a doctor to be ashamed of Andrew Wakefield, but maybe most of them aren't. And even if they are, it could be a deep psychological issue that I choose to compare the state of our industry to what Wakefield did in his.

But here's the thing: I know that we can do better. Not perfect, better. But we don't have to, because we lack accountability.

Sure, perfect is the enemy of the good. But complacency is the last refuge of the mediocre.

> Ashamed that someone else's system doesn't work? I can't help you there. That's some deep psychological issue.

Personally I get angry in those situations... the larger the company, typically the more safeguard crap in place that's supposed to prevent those issues, the more pissed off I get. But that's just me. I get angry when I see evidence of poor/buggy quality in business software the more visible it is.

>This mentality is why I feel ashamed every time a receptionist apologizes for the wait

why would you feel ashamed for waiting a little longer?

that's an odd thing to be insecure about

Because the guy works on computers for a living.

I'm not ashamed for waiting longer. I'm ashamed for the same reason I would expect doctors to feel ashamed about Andrew Wakefield.

I have a feeling you dont have much experience on working on other peoples code. You have just bundled all kinds of issues into a bucket of "buggy piece of poorly written code". Some of these are outright security vulnerabilities and some make you wonder how was the bug not found so far. Sometimes fixing these issues require a heavy rewrite which cannot be justified as a bug-fix. And how do you justify playing with your customers data by "slapdash jquery-and-duct-tape" techniques.

"Software should really only be measured by the value it provides. If a terribly buggy piece of poorly written code still saves hundreds of man hours a week, it's a win. (Unit tests be damned)"

But those bugs could easily lead to mistakes in the output.

Really? Ship an app to the store that crashes all the time and see your metrics go WAY down. Loosing user data is another great way to lose users permanently. Sure if you are Google of FB it probably does not matter but for smaller companies NOT CRASH, NOT LOOSE USER DATA seems to matter quite a bit in my experience.

B2C is a tiny, tiny fraction of the overall software industry. The vast majority of developers are working on godawful enterprise software that end-users have no choice but to tolerate. The absolute worst apps on the iOS store are remarkably high quality by the standards of enterprise software.

I suggest reading the comments on this thread:

https://news.ycombinator.com/item?id=13859961

Spot on in my experience. Curious if anyone at Google/Facebook/Amazon/Apple core teams could comment on whether these issues are also the case at Google/Facebook/Amazon/Apple? I would imagine solving issues like these do make business sense at big firms with scalability problems.

This will be changing as software is responsible for the control of an increasing number of physical systems (cars, medical, aircraft, etc.)

Edit: Unless we elect people who take away legal protections in place to regulate safety critical software

I had the fortune to work at an agency that cared a whole lot, and tended to take on clients who had learned hard lessons from bad code written for the platform we worked in.

Code quality matters and has tangible consequences in that niche, and it was awesome to work in that environment.

I haven't worked anywhere like that previously, and I think the key factors that brought code quality to the forefront were:

* Bugs cost sales

* Poor code caused friction for features a significant amount (reduced ROI)

* Small teams and an insanely tight customer relationship to those teams. The client needed to have a stakeholder in our team or as close to that as possible.

But most importantly it was just the company culture from top to bottom. The director cared, he made the project managers care, and we already cared because it's more fun to write good code.

That's a nice place to work in! How was this company doing? Would it be doing better just shipping "if it makes money - it's good enough" code?

It really was! I believe they were doing well, growing at a steady pace. Another part of the puzzle was that client relationships were long, over years, so if we wrote bad code it was us that would have to deal with it later. The clients were typically larger, which helped to that end.

We did take on a rushed project at one point, so there is a good comparison in that project. It shipped, but everyone was unhappy, some key people left and had to be re-hired for. It triggered a very transparent discussion roughly titled "let's figure out how to never do that again" and to that end they curated the clients more carefully, and put in some checks to see if projects are a good fit.

But I understand that is probably unique, and at previous jobs the "good enough to make money" mantra has worked out okay.

I think when your software domain is complex and rapid, it pays to have sound architecture. When it is simple, or slow to change, it doesn't matter quite as much.

The industry does itself no favours by regularly rejecting techniques to improve software quality tho'. We could all be writing rock solid Ada, but programmers said no, we prefer JavaScript, hell we'll even try to use it on the server...

Lack of profit focus is definitely something I see in junior devs of the higher quality kind. It's important to sell any prospective improvement to management in a way that makes money. Quality people can't see won't make money, and even quality they can see will only make money if there's a competitive market.

Do they care if a year down the road your maintenance costs increase fivefold? Have they seen the analysis which shows that initial development cost just makes a small fraction of the total cost and that fraction get smaller the longer software is in use?

Do they care if a year down the road your TTM times increase tenfold?

  We know that the bad code slows us down. Why do we write it?
  Why do we write the code that slows us down? How much does it
  slow us down? It slows us down a lot. And every time we touch
  the modules we get slowed down again. Every time we touch bad 
  code it slows us down, what do we write this stuff what slows 
  us down?
  And the answer to that is: well we have to go fast.
  I'll let you to deal with the logical inconsistency there, my 
  message to you today is—you don't go fast by writing crap.

https://www.youtube.com/watch?v=TMuno5RZNeE&feature=youtu.be...

No. When they have seen all the bugs, they will tell you to fix them and when you make a new feature they will tell you the same thing, just make it work, just ship it.

if you improved the efficiency of a hpc shot by 30% I am sure people would sit up and take notice :-)

>as long as it does not kill the customer

Whoa now. The Toyota 'unintended acceleration' case showed us that it's fine to kill the customer. The court declared themselves incapable of penalizing even the most stupendously egregious example of criminal negligence we're likely to see (Of 90+ 'required' and 'suggested' practices in automotive industry firmware coding practices, they did 4 in the code in question. The developers didn't have a bug tracker or static analysis tools. Typical of all organizations, the software engineers had no real power to demand more time or testing, etc). They lost a civil wrongful death case and paid out $4 million, then settled with the plaintiffs out of court before the jury could award punitive damages. So killing is OK.

The issue is that, I think, software people need to stand up for themselves. They need to tell executives no, and refuse to pump out slapdash work. They need to take responsibility for the successes and failures of the company. The practical truth is that in most organizations, the software is the most valuable part of the entire business and the thing with the power to enable the company to excel or fall flat. We've been waiting for executives to recognize this since the 1980s and they still just see the software people as people who do typing.

Totally agree. But the only way for software people to take any power or control (really, for any worker to have any say in the business at all) is to unionize, and us tech folks tend to treat that like a curse word.

Union is a curse word but I have a hunch that there are others that might make more sense. Guild maybe?

Bad example.

After a 10-month search, NASA and NHTSA scientists found no electronic defect in Toyota vehicles. Driver error or pedal misapplication was found responsible for most of the incidents. The report ended stating, "Our conclusion is Toyota's problems were mechanical, not electrical."

https://en.wikipedia.org/wiki/2009%E2%80%9311_Toyota_vehicle...

> The sooner you accept this, the less likely you'll burn out.

I learned this at my first software engineer (though we didn't call it such) job back when I was 18; closing in on 45 years of age, and I'm still here at it, and don't see that changing soon. I have thought about management, but I really don't have such experience, and I'm not a real people person anyhow - plus I like to code.

Another thing I have learned - at least here in the United States: Not only do they not care about quality, they usually don't have any loyalty to you as an employee. They'll let you go without any warning at all, and if you're lucky, you'll get some severance pay.

So save your money, build a savings account with FU cash in it (6 months to a year of salary - or more), and don't be afraid to drop an employer like a hot potato if things aren't working out, or a better offer is in the works.

And above all else, don't let your skills stagnate.

I think you've nailed it on the head but haven't realized why you've nailed it on the head.

`Why? These things have real consequences.`

To a business they see software/engineers as a expense. The whole focus as a business is to provide a service that people are will to pay more than it costs to provide said service.

Everything else doesn't matter, what does matter for a business is $$$$$ and making sure business doesn't get sued because bill touched jill with his #####.

For what my words are worth, I do think you need to take a step back and view it from a Business person perspective and attempt to align your goals with theirs. Get out of your cube and go and meet CEO's and CIO and talk to them about their problems and what their goals are. Then see if you two can work together to achieve their goals and bring value to the table.

Even writing this I've could think of 10 or 20 different business discussions you could have.

>To a business they see software/engineers as a expense.

Anecdotally, this is one of the first things I try to gauge at a prospective employer. "Are their development teams seen as cost centers, or profit centers?" If the company works with technology a lot and doesn't see technology as being particularly important to its bottom line, that's usually a bad sign.

Also, discussions with C-levels tend to be pretty one-dimensional. It doesn't matter how much their infrastructure is falling apart, they'll still stonewall with something along the lines of, "our goal this quarter is new user acquisition - how does this improve those numbers?"

And they won't accept, "users cannot sign up if everything is broken" as an answer.

You're 100% right but also 100% wrong.

Like I said before you're viewing it from the perspective of a developer but not from the perspective of a Business person.

Instead of saying `Users cannot sing up if everything is broken` you need to see it from a Business perspective, and communicate it to them in the language they understand.

For example, `From our quarterly earnings report we need to have 10,0000 new clients signed up by the end of the month to reach the goal. We have on average 2,000 sign-ups per day. On the 2nd, 3rd of last month two of the critical system's fell over, and this result in a estimated 4,000 failed client acquisitions. The estimate from our quarterly earnings each new customer bring's in a revenue of $300 net profit from each client for the quarter. As a result of the system crash our quarterly earnings will be down 1.2 Million.`

Your job is to communicate clearly in the language they understand how it will achieve their goals. You're not the only department/manager who is requiring resources from the Business. As I stated before you're there to serve the business to achieve their goals.

The corollary here that it is perfectly acceptable for business users to not understand significant parts of their own business operations. In fact, everybody should work around your shortcomings in understanding that "technical" stuff. As the guy with the MBA, you are the most important person in the room and everybody else is there to serve you.

You are a team. Not everyone has the same background and experiences. You would be amazed at how many people in an organization have incredible knowledge of their business domain and little to no knowledge of IT systems, their costs, and benefits. As an IT leader it is your responsibility to help bridge that gap, and the language of business is business not tech jargon, so I would caution against dismissing it.

Another way to think of it is as of user acquisition problem. You have to highlight the benefits in a way that is compelling to them. At the end of a day all progress/fiascos in a company comes down to consistent and effective communication.

Still doesn't work. I've done clandestine work over weekends to speed up an application, then as part of shipping it also add a split test with half the audience having a delay similar to what was in place before I fixed the problem and brought that to the table with the "business" guys, and everyone nodded their head "wow" but every sprint after that was just full of new features.

I hope you got the lesson. Please stop working over the week ends for free.

You are devaluing yourself and you are devaluing all the developers who don't want to ruin their week end at work.

Agreed, but don't take this as "don't work hard," merely as "don't donate time to your employer." There are far more worthy causes out there.

I obviously didn't bring to the demonstration that I had volunteered the time, it suited my point to make it seem like it was something easy I could do as an aside... Problem is, they don't care about $ either, they care about what makes them look good, what makes their boss look good, what the sales guys are bugging them about, and it's features and visual polish and never quality or performance.

I had to break it down to one of them "Listen, we've seen that the site being faster gets us X% more conversions, if the site was down, or if the site was ugly X% of the time and cost us those conversions, you'd be breathing down my neck to fix it, why is this any different...?" reply after a few seconds of thought "You're right, but you'll never convince Leadership"

> it suited my point to make it seem like it was something easy I could do as an aside...

Thus fuelling the idea that this sort of thing will just be done as other work is happening with no time budgeted for it.

There is an old Google paper where they talk about discovering that users returned more often if the search results list was 10 instead of 20, since the shorter list took marginally less time to load/render for the user. If your co-workers don't believe you then I am sure they would at least love the Google finding that minor performance improvements have a positive impact. They would then support improving performance in the app and brag about how much like Google they are.

>>> There is an old Google paper

No, there isn't.

Google is showing one series of ads per page. The study showed that having more legitimate results per page decreased their revenues. Of course, it gives less room for ads and less page views.

I saw your post but didn't reply.

The work you did probably is great and do I think its above your average developer. Somewhere in the the top 10%. Though how you conducted yourself in the face of management was down right petty/folly.

Yes you did the best job in the world. You improved the performance of the serves by % and it did make management of those server's easier. Though you need to think about what you've done in the eyes of management.

Firstly you've modified the server's without permission. You've run tests out-side of work hours that could of impacted the business bottom line. For all I know (its up for debate) but your senior managers had more pressing issues on the table, and having a young code money running around outside office hours is the LAST thing they want to hear.

`It's not a simple as you may think it is.`

Some of the things that come to mind, and honestly I would of pulled you into a office to explain yourself if you're on my team.

1) Did you document all the changes you've made to the server

2) Did you back-up or store any sensitive information off-site during out of office hours

3) Did you modify critical system files that could cause another dependent system to fail

4) What ETA/Service level agreements if any did you break when bringing down the server for maintenance?

5) Did we lose any Security Certification with you modifying the server or changing any of the certificates on the server?

We're software developer, and yes when we see problem's we want to fix it to our best of our ability. Though when you're in a Business there are many thing you may not be aware of. In this situation I would honestly say you're lucky to keep your job! I hope you take it into consideration next time you want to go out-side of your duties.

I don't want to come down harsh on you, but there are times when you see a bad situation and there is nothing you can do about it. You notify the people who're higher than you of the situation and the dangers. It's then up to them to make the decision. After they've made the decision that is it. If the server blows up during the weekend, and they want you to come in and fix it (even when its there fuckup) you just take your hourly rate and x10. So if your hourly rate is $50hr and the server blows up, then your new rate is going to be $500 per hour to come in during the weekend to fix the mess.

I have a counter-perspective to provide:

Many of the points you've raised are moot outside of large corporate environments and projects.

It is likely that the commenter has root access to all of the things you mention and touches them on a daily basis. It is likely that they are not inundated with all of the heavy process you're bringing up. It is likely that all of the security concerns you raise are not even part of their current business. Working outside of set business hours is likely not a concern; overtime is likely uncompensated anyway and there probably aren't considerable office security protocols (they probably have keys and an access code for the whole office).

The points you've raised do not apply to most small development teams.

Yes, but both of these viewpoints are important, depending on the context.

It was a simple performance bug (think N+1 query type) that was trivial to implement and trivial to be confident about its behavior vs the pathological implementation, and I used the normal channels for shipping it, it passed QA.

The only "clandestine" part is that I added the split test, and did some work outside of the set of features that was scheduled.

We had no policies around SLA, developing off-site, security or restricting sensitive information (think potential FERPA/HIPPA violation) which are yet other "non-features" that I tried to get them to adopt.

I implemented this entirely above-board, with more precautions and consideration than were taken by other developers on that team for the day-to-day.

> I've done clandestine work over weekends to speed up an application

> I obviously didn't bring to the demonstration that I had volunteered the time, it suited my point to make it seem like it was something easy I could do as an aside

I understand the impulse, but in long term this leads to inflated estimates from management - and tech side has only themselves to blame. When we pretend things take shorter then they really do, dont be surprised when they learn to expect our lie to become truth.

Long term here can be as short as three months.

So we just have to do their job as well as our job?

Making money for the company is every employee’s job.

So the janitor should analyse and quantify what difference he will make to the companies bottom line before he wipes those skidmarks off?

Not everything is quantifiable like that. If I'm writing code and I take the time to make it 30% faster then it won't make a difference to our bottom line. If that's done everywhere then customers will like us more, be more likely to stay with us and be more likely to recommend us, but that can't be quantified in my day to day work.

Probably the biggest change looming over our industry is coding to - and deployment on - platforms that definitely are instrumented to capture exactly how much value a developer is adding (or subtracting) from the bottom line with every commit (it's a natural outgrowth of Function As A Service architectures).

It is already possible to more-or-less show how a code change (bug/fix, new feature, change in storage backend, etc.) directly affects your IAAS bill at the end of each month, and taking it a step further by tying in some instrumentation and formulas (of the sort often used to calculate the effectiveness of marketing funnels for optimization and A/B testing) to show the concomittant effect on revenue allows ROI to be calculated.

So, I am pretty sure that Finance-Oriented Programming is going to be a thing not so many years from now, tied into the whole toolchain, and very few - not developers nor their managers - seem prepared for that sea-change (some Operations folks are probably a bit better off in terms of mindset and processes).

The core conflict is this:

The engineer is trying to optimize/normalize his free time by acknowledging quality issues, the management is trying to optimize their income by sweeping them under the rug.

The engineer is ultimately liable for the problem. Quality is insured with his reputation and free time.

The business guys almost always leave at 4:00pm and their bonuses are based on a metric not related to quality.

Honestly, this is pretty straight-forward stuff, and I'm surprised you're getting any argument. If a technologist cannot explain how a proposed implementation is going to provide value to the business, that project is probably better off left alone. Prettier code and better tests don't provide value. But fewer customer-facing P1/P2 tickets provides value. That prettier code and better tests leads to fewer tickets is an implementation detail.

Let's put it this way: Netflix has one of the most robust microservice architectures I know of. I promise you will find zero mention of it anywhere on their customer-facing pages. Because that is not the problem that Netflix solves for their customer; they solve a media library and streaming problem.

The business is your customer. You need to understand their problems and how you're going to fix them. Everything else is an implementation detail.

>Honestly, this is pretty straight-forward stuff, and I'm surprised you're getting any argument.

I'm not. Our industry is full of academics who've been told "this is the right way to do it, everything else is stupid" for the entirety of their education. They're academics, trained by other academics, ignoring the broader reality of why we do these things at all.

>"Are their development teams seen as cost centers, or profit centers?"

It's not a matter of opinion, but what the term means. They are cost centers unless they are closing sales (or being billed for about 3x what they cost, which is still way less than sales people will generate, being closer to 10x+ cost).

Best thing any employee can do is find out how their company actually makes money and understand how they align. Unfortunately, the highest business expense is salaries and one of the most important things for most businesses is remaining runway. So there is huge pressure to keep salaries down, particularly in cost centres, which aren't directly extending runway. If it's a service business it can get worse, as the money is typically made on billable hours - creating a perverse incentive to have cheaper and less efficient people to bill out rather than a 10xer (outside a senior business facing role). As long as they are capable and not so bad that they lose the client, it's often more efficient to have cheaper mediocre people from a business point of view.

How can a software developer fight against this salary limitation?

Become a highly sought after specialist for a product offering (e.g. self-driving cars, blockchain, fintech), or get ownership (startup), or go customer facing (particularly if commission involved), or go into technical management on CTO route.

> one of the most important things for most businesses is remaining runway

Remaining runway is a measure that only exist for companies in the red. For most business it doesn't even exist.

How long you can continue operations if sales drops off a cliff is a very important number for every company I've worked at, from startups to big multinationals. Maybe it's less of a consideration for Amazon and Apple but I wouldn't even bet on it there.

The better question to ask in my opinion is whether or not they activate their IP on the balance sheet. If they do there is a very good chance that they care about the quality. If they don't then there is a very good chance they treat software as something disposable.

> Get out of your cube and go and meet CEO's and CIO and talk to them about their problems and what their goals are.

Jesus Christ listen to yourself. You don’t know anything about me or how I approach things with my clients.

I care about how code affects humans but business don’t have to be responsible for how they hurt people with it.

> Get out of your cube and go and meet CEO's and CIO

Oh right.

Good luck even ATTEMPTING to arrange a meeting with a Fortune company's CEO. "Who from what department again?". Worse yet if it not even your company's CEO.

Maybe, if you are lucky, you can get to skip a level or too. Software engineers are usually very far removed from the top leadership.

I routinely talk to CIO’s. They’re often the ones who hire me.

Yet my experience is that the issue comes from engineering a lot of times: fixing bugs is not fun, let’s rewrite everything because this time we learned from our errors, or just to use the new shiny framework. A buggy legacy code base that no one wants to touch or improve, and V2/New projects that of course completely fail to deliver in the end and the cycle repeats. Being in a customer team I try everyday to make Engineering care about non technical end users. My team and business might be responsible for some feature creep, but bad code quality comes from engineering practices.

Edit: to be more clear the discussion usually goes like this:

- Me: “where are we on fixing this bug? it’s been two years and customers are still complaining”

- Engineering: “we can’t touch at this component it’s a mess but we are rewriting it, this bug will no longer exists and it’ll be so much powerful, just wait a few month”

We all know how it ends. And when business pressure comes because it failed to deliver it’s all “business is evil we need more time to do great software!”

What we really need is senior and experienced engineers IMHO

> Get out of your cube and go and meet CEO's and CIO and talk to them about their problems and what their goals are.

Implied in this statement is that the parent has a myopic perspective on the business world. Based on my read of his comment, I don't think that's fair - and moreover would question whether or not your perspective is as unique as you apparently think it is.

In other words, I think you're patting yourself on the back a little much for figuring out how to chew before swallowing.

Regarding Equifax, when a company like that gets enough negative public reaction, they look for a bandaid. In the case of Equifax, they recently hired ReliaQuest https://www.reliaquest.com/

I've friends at ReliaQuest and I know it is a good company, but I am frustrated that the leadership at Equifax thinks that security is a minor issue that they can outsource to a 3rd party. If a company deals with sensitive financial data, shouldn't security be a core competence?

Reply on news.ycombinator.com