Hacker News Reader

Comment by whack | Hacker News Reader

To me, this was the money quote:

> You might say, “But event sourcing is so elegant! Having a SPA on top of microservices is so clean!” Sure, it can be, but not when you’re the person writing all ten microservices. It is that kind of additional complexity that is often so unnecessary.

It's very easy to be wasteful when:

1. It's not your time that's being wasted

2. It's not your money that's being wasted

One of the best learning experiences I had, was building and launching an entire application API, purely by myself, in my free-time while juggling a full-time job. It really helps you accurately evaluate the cost-benefit tradeoffs for various flavor-of-the-month technologies/methodologies, when you're the one who has to shoulder every one of those costs.

I've had far too many team-leads whose assessments of these cost-benefit tradeoffs are intensely skewed by the fact that it's other people who have to work on those things, and it's other people's money that is being burnt while working on these things. We think these are technical debates, but really, it's a organizational problem. Under the right organizational structure and incentives, people are exceedingly bright at optimizing for their personal success. The principal-agent problem is a crippling handicap that hobbles every medium/large business today. The day we find better ways to solve this problem, is going to be a transformational day in the history of our civilization.

https://en.wikipedia.org/wiki/Principal–agent_problem

View on HN

If you're building something you plan on throwing away, or won't be around to maintain, it makes perfect sense to throw things together quickly. If you're building something that will be a foundation for many future projects, however, it's worthwhile to put more thought, and thus more time, into building it.

> It's not your time that's being wasted

More frequently than I would like, I find myself putting a band-aid fix on something that I should have spent more time thinking through up-front. In this case it's future-me whose time I wasted by not taking my time initially.

In a startup scenario, business changes can and will come in that destroy previous assumptions. Being stuck with a 1/4 finished cathedral when you now need to build a truck is both wasteful and inneficient

Being sloppy and being fast are not the same thing.

One can be fast by limiting complexity, yet provide a disciplined implementation.

In an uncertain scenario, such as you described, I would say it never pays to be sloppy, but it often pays not to do anything more than is absolutely necessary for the time being.

Ah the million dollar statement. Ofc you can make something completely right, that ”race car” project but then someone says “but I want a bus” and you transform it as fast as possible to a bus and somewhere half way through someone comes along and say “why isn’t it flying”.

I've never worked anywhere in 18 years where the goals of a project weren't at least limited to the correct 'vehicle class'. Implementation details sure can change but that's why "fast and disciplined" is good - you learn to implement the software in a manner where you understand the changes needed based on concrete principles rather than a mad dash of copy-and-pasting.

By 'disciplined' I don't mean that the program fullfills the dogmatic mandates of some system development philosophy - just that the software is written on solid principles that at least the author understands.

Just being curios, have you been working on the same platforms/systems during all these 18 years?

Both what you are pointing out and what the other commenter are pointing out are critical, contrasting failure modes that must be continually avoided. You both are correct.

The balance between the two will depend enormously on the circumstances, and is difficult to manage.

Couldn't agree more. The bandwagon effect is something also worth mentioning. Tech stacks and tools these days are branded almost as consumer products and the outcome is poorly designed systems.

The net present value to a developer of dooming their current employer's current project by choosing an unsuitable technology which they could put on their resume to negotiate future salary (at a position they'll be moving to in, on average, months not years) is going to swamp betting on something boring but stable, completing the project, staying put and crossing their fingers for raises that outpace inflation.

The root problem here is our hiring and interviewing system.

Imagine if interviewing rewarded "look I know all this stuff about writing simple functions in Stack X" less than "yes, the project shipped, and beat its deadline, because we decided to use library Y that the devs already knew even with the acknowledged pain points, instead of bringing a new technology in just for this project. We have some less-mission critical areas where we decided to experiment with new tech, but needed something predictable here."

Evaluating a senior dev based on junior-dev-esque implementation tasks instead of "does this person know how to avoid over-complicating our job?" is ludicrous. As an industry we interview to find "clever" people rather than "wise" people.

Outside of web development, you can get a job fine without using this year framework. However, you still have to leave after a year to get more than inflation.

> you still have to leave after a year to get more than inflation

This hits home. I've been in software development for quite some time (10 years), I've been with 4 employers so far (full time) and pretty much all my significant raises have come as a result of me looking for a better paying position and then leaving. I may have been unlucky (I also did not work for a really large company yet), but I feel like the "career develoment" opportunities within software engineering companies are way under the level they should be.

I think this has been the way of the tech industry, and particularly SV, for a long time now. My uncle, now passed away, started at Atari in the early 80s and hopped companies every three years until his final job with Nvidia in a fairly senior position. He stayed with Nvidia until his death a little over a year ago.

Each time he switched it up he grew his salary exponentially.

Exponential growth, every three years, since the 80s... sounds a bit hyperbolic, doesn't it? Just how much did he make at Nvidia??

Exponential growth means just constant growth rate. Doesn't mean it's a big growth rate. 2% a year will double the starting salary in... 35 years.

;).

No it doesn't. It means an exponent is present.

https://en.wikipedia.org/wiki/Exponential_growth

The formula for constant growth over time, for example, a 2% increase every year, would be starting salary * 1.02 ^ t where t is the number of years. So there is an exponent present, and constant growth is an exponential process.

Constant growth rate gives rise to a quantity that is an exponential function of time [not to be confused with a constant rate of increase, which gives rise to a linear function of time].

As that Wikipedia article says, the continuous-time equation for exponential growth, x(t) = x(0) e^(kt), arises as the solution to the ODE x'(t) = kx, where k is the constant growth rate.

Similarly, in discrete-time, exponential growth follows the equation x_t = x_0 (1+r)^t, where r is the constant growth rate.

I stand corrected.

Maybe Nvidia was his second job?

That is pretty much country specific.

Been with the same employer for several years, jumping between web and native projects in all those years, across multiple clients.

Inflation was never a problem.

The problem in my experience working in "enterprise" shops is that more often that that, most of the employee hierarchy is in on the scam - managers know developers are padding hours, developers know middle managers are peddling snake oil up the chain, and senior managers are painting a picture of success to their superiors to get their big fat bonus. And very often, almost all of it is an illusion, and when everyone's done and the final product sucks for end users, everyone responsible has long ago declared it to be a smashing success and moved on, and the poor support developers have Yet Another Over-engineered Mess on their hands. Rinse, repeat.

The purest capitalistic endavours are drug deals, theft and scams.

The more refined variants, which find law loopholes and apply them with leverage,are still the same approach.

Look at those jailbirds under the sky, they do not sow, but our Lord the Ford, supplies them better then those working none the less.

The shareholders of a company know this deep down as well, so one serial scam series later- the comon pension er who had to use bad software all his life, can retire knowing he ripped off some poor schmock to have this.

My experience in Australia is this is OK. Very few companies for example would expect you to have React experience. If you have any web framework experience (or just know JS well) you are OK.

Usually on the back end they are fussy about your main stack, e.g. C# -> Ruby would be hard. Java -> C# more possible. However within that, which frameworks you have used, e.g. Akka, NHibernate etc. doesn't matter.

NoSQL, or any particular DB technology is rarely expected.

My experience is companies are fairly conservative with their tech choices here. A 4 yr old framework is typical! Usually there will be 1 or 2 cutting edge things being tried out, but without uprouting everything.

THAT RIGHT THERE.

The newer development trends haven't focused on experience. They've focused on resume driven development.

It's not a tech issue, it's a business issue. Why give people bigger raises than they may otherwise ask for? That's just wasting money. It's the same logic for why you only get a good deal when you threaten to leave your mobile phone provider. They don't want to proactively reduce your rate or make it too easy to get a discount otherwise they might be doing it unnecessarily.

The squeaky wheel gets the oil.

With work, it's also about an employee's best alternative, and also the psychology of "well, you used to work for x, so we'll only pay you a small increase since that's obviously what you'll work for". It's as much the psychological aspect as anything else since it's difficult to decide the "price" of something without any reference (i.e. a new employee).

Therefore it's crucial to have a nice work culture. When employers only think of themselves, employees copy that. On the other hand the people doing the actual coding must also be able to learn something new, that's just how the technology world works. If the employers don't help with that, the employees are doomed eventually.

Exactly, frameworks and tools are resume driven, everyone is putting new ones as a means to sell themselves to companies, do conference talks, set up their consultancy, ....

And when they move along to the new shiny thingy, those tech stacks die unless they achieved momentum beyond the usual adoption curve.

Amen. I think the effects can be significantly lessened though under various circumstances:

- highly technical boss (understands that things might backfire)

- flat hierarchies (responsibility isn't delegated through layers...)

- reasonable work culture with nice people

But yeah, as an employee this is a question of luck, landing in the right place.

I think you're on the right track when you're saying that this isn't a technical problem. That's the most important thing to understand about the development of overwrought, top-heavy system architectures.

Trying to fix this with more code is a fool's errand. As soon as you get something reasonable and good, someone will come along to distort and abuse it to serve their non-technical interests.

IMO a few of the main reasons we see this plague of listless over-engineering are:

a) many people don't actually know what they're doing, and they're trying to hide behind ever-shifting technical targets to obscure this fact. Sometimes this is pretty blatant, but this happens in a more subtle form with surprising frequency. Many people maintain only a very superficial interest and understanding in what's happening under the covers, so they hear promises of magic hand-waving and instantly jump to them.

and b) people are forced by the principal to coop themselves up in an office for 8 hours per day, and they need to invent things to occupy their time. You could say that this is the agent not respecting the principal's investment, but it really goes both ways. Our employment model does not allow for "done", because as soon as this happens, the contributor is declared redundant and the revenue stream disappears. It's fun to brainstorm models that may not have that issue.

A relatively flat/reasonable technical apparatus is needed to reason coherently about a system at all. A lot of these microservice shops are complete nightmares, and they use "microservices" as cover for "nothing is maintainable, just delete it and start over when we need to make an important change".

People primarily want to use the latest stacks and make complex system architectures to gratify their vanity, in various ways: to keep their resume relevant, to be treated like a guru, and to keep up with the joneses. These are problems that stretch way outside of tech and affect everything humans do.

Have other technical fields found ways of handling this sanely? What is stopping civil engineers from changing to a different concrete formulation every two weeks? Is that there is an objective physical component that can be tested, whereas software lives primarily as a manifestation of a mental problem-solving structure? Those are the answers that our field needs to look for IMO.

> and b) people are forced by the principal to coop themselves up in an office for 8 hours per day, and they need to invent things to occupy their time. You could say that this is the agent not respecting the principal's investment, but it really goes both ways. Our employment model does not allow for "done", because as soon as this happens, the contributor is declared redundant and the revenue stream disappears. It's fun to brainstorm models that may not have that issue.

I think you might be overstating this part. I haven't been in many positions where if I said "hey, I finished that thing you asked me to do, and it took five fewer people and six fewer months than we thought" the response would be "ok, looks like we can fire you all now" instead of "great! get started on the next from the list of 100 things we think might be useful for the business!" + some good results at performance review time.

My usual frustration is the opposite: I can't knock out my tasks as quickly as I could because I'm waiting on other teams to figure out how to add an unexpected new feature to their overengineered buzzwordy codebase.

Maybe this is a high-growth vs low-growth environment type of thing, but even in more stable businesses, seems like the maintenance backlog never really empties. Just gotta make sure the bosses understand that you wouldn't fire the workers every time the shop temporarily clears - you're still going to need them for the next jobs that come in.

I think that a lot of the motive in this direction is unconscious. It's less "Oh boy, they'll realize we aren't doing anything if we don't say we need to rewrite something", and more, "This will be an interesting way to spend my time and keep myself busy for a while." This is the same impulse, just vocalized differently.

If the incentive structure was changed, devs may rather think "I need to make this as tight, small, and maintainable as possible, so that I can spend the rest of my time on the beach." We see this a lot more on side projects specifically because the dev knows and respects that the available resources are constrained, as the parent poster mentions.

In any case, people usually won't allow it to reach the inflection point where they're visibly non-productive and the boss is cognizant of the option to "clear the shop".

There are always tweaks, defects and flaws that can be made up and justified. The question is whether that work is really contributing useful value. Once a project reaches a certain level of maturity, this becomes increasingly dubious.

That's your team's problem. Optimizing single developers for "getting their part over" is not contributing. Software delivery is not like taking tests in school. It's team delivery over multiple release. The only wins are group delivery.

>nothing is maintainable, just delete it and start over when we need to make an important change

Good! That's why we have microservices, classes, or interchangeable parts in any engineering discipline: rather than laboriously extricating a deeply integrated, tightly coupled component (or worse, give up and throw away the whole system), we get to rip out one specific component, throw it away, and replace it with something that conforms to the same interface while keeping the rest. If you can routinely delete and replace specific services, your microservices architecture is working beautifully. Just as, when you can routinely overhaul specific classes, your OO design is working well.

In the physical world, maintenance and repair hardly ever consist of coaxing the same matter into working state. It's extracting old parts, throwing them away, and putting in new ones.

> In the physical world, maintenance and repair hardly ever consist of coaxing the same matter into working state. It's extracting old parts, throwing them away, and putting in new ones.

The difference is what you consider "the same matter". Since software exists in the realm of the mind, you have to come to an explicit consensus; there is no natural or physical barrier.

A mechanic will allow dirty oil to drain, but they won't rip out your oil reservoir and redo all your hoses every time you want to get your oil replaced. That'd be very expensive and wasteful. It is not good to use parts that are so cheap that they tear whenever someone pokes or nudges them. That'd make it so no one could move around without getting lots of oil all over the place.

Basic maintenance should be possible. In almost all cases, you should not export every function as its own independent microservice. You should not have to track a request across 60 internal services when you're trying to debug. Yet, this is what we've gotten from naive microservice implementers all too often.

In a microservice architecture, many of the tools that would otherwise validate and verify a successful refactoring are unavailable, because dependent code is exported and accessed over HTTP instead of language-level constructs that the compiler or other static analysis tools could check.

In any architecture, it is rare for a programmer to be able to gut a dependency without causing collateral damage somewhere down the chain.

Somehow, this policy of "throw out everything related to SERVICE_X and start over whenever we need to make change" doesn't seem very practical...

Microservices are never deleted. They are only abandoned and left to rot after the initial developers move on to their second system (https://en.wikipedia.org/wiki/Second-system_effect). Then you have apps that are stuck in migration purgatory, where they haven't fully switched over to the new microservice because the business doesn't want to spend that effort, but they do want developers to focus on using the new microservice and new API for new app work. The hope is that app will eventually be 100% on the new microservice, but after 3 years you start to realize that is never happening and by that time the company has an entirely new staff working on a much newer microservice that promises to fix all the problems with the prior microservice.

That microservice kool-aid is strong and deadly.

> replace it with something that conforms to the same interface

ha. hahaha. Oh my. I'm in the process of switching over a REST service to a GraphQL service. Because apparently people wanted all the pain of SOAP, but "could I get that in a JSON flavor, please?" Five years from now, REST will be making a comeback under a different name. You heard it here first.

REST failed at marketing itself and its primary document is a doctoral thesis, hardly something that an average developer is going to grok, understand and implement.

Hypermedia is a more appropriate term, and emphasizes the actual substance rather than style.

The idea of non-architectural changes to software is a dangerous myth. The "architecture" should reflect the problem, and when the problem changes so does the design. You almost never want to replace a component without changing the interface.

Caring about those decisions even when it's not your time or money is what I think "craftsmanship" is.

What you say is all true, but its also a local maxima that we're all caught in. I think there's a critical threshold at which we could achieve accelerating good with computing instead of today's computing doing a lot of harm. What the path to that threshold is, is an interesting conversation to have.

A few decades of observed reality suggest the path involves reversing the arrow of time. Software fucks up everything it touches in one way or another, with few meaningful gains outside of simple convenience.

At the risk of going off on a tangent, your pessimism is unwarranted.

Consider, first of all, that we're having this discussion on a message board with participants from all over the world. That's a meaningful benefit of software.

I read both the original article and this discussion with a screen reader, because I'm visually impaired. I can also use a screen reader to read ebooks. Sure, I have enough sight that I could read regular print with a simple magnifier. But without software, my totally blind friends would have to wait for the things they want to read to be transcribed into braille, recorded, or read to them in person. This, to me, is a benefit of software that goes way beyond convenience.

I could go on, but this blog post by Bertrand Meyer responds to general pessimism about software better than I could: https://bertrandmeyer.com/2013/03/12/apocalypse-no-part-1/

Spend some time contemplating all of the ways that the modern surveillance state oppresses minorities, all of the drivers of income inequality (both domestically and globally), how few people actually benefit meaningfully from automation, and get back to me about how my "pessimism" is unwarranted. FFS, software has lowered the barrier to warfare so far we're bombing somewhere between five and ten (I can't even keep up) countries currently, and it isn't even generating column inches in the local paper.

Modern states deliver drastically higher standard of living, less oppression, and minuscule existential risk (i.e. total war, wholesale slaughter) compared to historical norms.

Read some history and get back to us on how the modern world is oppressive, violent, or impoverished compared to some bygone golden age.

It’s true that if you go back far enough we were all equal... because we were all peasant farmers.

I've listed observable deterioration in societal conditions attributable directly to software, and thus changes that have taken place over roughly the last 50 years. Are you asserting mass surveillance (for example) hasn't become a largely inescapable societal norm or that software doesn't provide the methods? In what way is "historical norms" before the advent of software in any way relevant to the discussion? To humor your strawman, please reconcile the phrase "miniscule existential risk" with the myriad existential threats to species survival the modern age presents that did not exist before the advent of computers.

The surveilance works also in reverse. Black Lives Matter wouldnt be there if there wasnt the masses surveil the supressors.

Alot of technology benefits are non visible today- but imagine any nasty event- and notice how a lot of the implicit knowledge of a society would survive today- encoded in videos, automated tutorials and wikipedia articles.

And it even makes kind of sence. Investing into "Leaps" forward can fail in a thousand ways, investing into measure that prevent society from falling down completely and allow fast recoverys, well that is something that can become usefull a hundred times over.

Its like a dice where one in nth sides gives you a chance to leap forward - and 0.5*n of all sides have a chance you must step permanently back. Removing those sites of the dice from the scenario tree, is statistically OP though not really cool to watch.

The only thing currently not OP is how companys research is directed towards local optima. There are no big attempts to lift whole areas of endavours in feasability.

Actually, if i had the whole picture- i would guess that humanity is currently on some sort of golden path.

> Alot of technology benefits are non visible today- but imagine any nasty event- and notice how a lot of the implicit knowledge of a society would survive today- encoded in videos, automated tutorials and wikipedia articles.

Let's be fair here. If a nasty event caused us to lose power grids globally for a year, we'd lose 99% of that knowledge (preceded by mass starvation of high-double-digits of the western population). Digital data is fragile. Hard drives have lifetimes measured in years. Even optical and flash drives have practical lifetimes (in actual use) of under two decades. The whole thing works only because data is not at rest for long - it's get copied over all the time, as our economy pumps out hard drive after hard drive after hard drive (hell, the same thing applies to CPUs, RAM and displays these days). Everything is so tightly interwoven together, that it starts to resemble a living being - you can't pause it.

That's, BTW., the reason I strongly believe our global priority everywhere should be preservation and stabilization of the technological civilization we have. Itself, it's not rebootable - if it fails now, if we get kicked back to pre-industrial-evolution levels (through e.g. 3rd World War, or environmental collapse, or both), we'll be stuck in them for millenia to come, waiting for Earth to replenish all the easily accessible high-density energy sources, that are all gone now.

We are on a golden path, but it's a tightrope - we need to focus on balancing ourselves and getting to the other end, instead of getting distracted by random trivia and amplifying every single thing that is not exactly right for someone.

I'm pretty sure the state was oppressing minorities and driving income inequality just fine without software.

If you believe that you're going to be sad when someone shows you a graph of income inequality over time. That oppression pre-dates software is not in question. Are you attempting to assert that modern surveillance and carceral state either don't benefit from software or are not examples of institutionalized oppression? You are, of course, aware that the US has the highest number (by count and % of population) of incarcerated citizens of any country on the planet, and that minorities are disproportionally targeted, right?

the web didn't. But your point about arrows is well taken. Today, the gravity arrow points toward the money. But I don't think that's some immutable law of nature, i think it's a local maxima (a very deep maxima)

> the web didn't

Oh, the web is likely the single biggest fuckup (maybe second only to C) of the software world. We've gained a lot, true, but we've also broke things a lot too.

Goes to show that "worse is better", given that these two perhaps badly-designed systems are the software world's biggest successes.

Yeah. "Worse is better" is just philosophy built around understanding of feedback loops (indirectly, through evolutionary processes). It's pretty much fact of the system that is human society. It's true, and it's the default way things work. It should be used, however, as a reminder that we have brains capable of overriding that default, if we can be bothered to use them.

Couldn't agree more.

On the mobile app I am working on, the hope is to move from dozen of Activities to a single one all MVI based.

That's nice (even though it will be a nightmare to debug and optimize) but it will takes tons of engineering work for absolutely not product return.

My current plan is to get the hell out of there before that.

What a brilliantly terse formulation of the topic complex that had me pondering for quite a while now! In some manner, "mulling over casually on-and-off" for 2 decades.

> Under the right organizational structure and incentives, people are exceedingly bright at optimizing for their personal success.

Optimizing organisations for converging towards such structure-and-incentives would be the money-shot, wouldn't it.. the only set-up coming to my mind proven to work (for team sizes greater than 1) are the cliche garage up-start and "small gang of well-aligned-in-skill-levels-and-obsession, ~90% of the work/design 100% under own control, product-builders". In other words, how can every small tech team feel like "early-days id Software" --- or rather, the perception of it that prevailed.

> The principal-agent problem is a crippling handicap that hobbles every medium/large business today. The day we find better ways to solve this problem, is going to be a transformational day in the history of our civilization

Indeed. "Medium/large" is worth repeating. Small shops need to get to medium to be able to financially and mentally afford to become some-degree-of-"wasteful". The question then becomes, why is this dynamic so predictably playing out every time that they indeed will? Is it human nature? A case of "everyone who continually succeeds in meeting their next goal eventually rises to their level of incompetence"? Is it that as soon as the adrenaline-rush of ensuring "the bacon" has actually worked, we naturally relax and become playful? With the only way to express 'playful' in a "serious setting" being to endlessly bikeshed and adopt new toys to play with, I mean "promising/proven tech to adopt and migrate/pivot to"?

Remember early-Google's (maybe they still have it) "20% of the week is play-time (at least for developers)". No (or at least reduced) need to channel the urge into counterproductive role-plays of long-winded 'architectural/tech/stack/paradigm debates'. No matter how you cut it, they must have gained heaps from this arrangement not just in product prototypes like Gmail, Maps etc but probably also simply in reduced time-wasting back then. I do suspect for 99+%-software companies, the most fruitful model might turn out to be like this but with 80% devoted to the "play approach" and only 20% of the week to revisiting the concern's concerns that inform and subtly (re)direct in what problem areas to "play more". It's just a bit too radical-seeming to invest a-million-plus in, unless it's one's own millions (and possibly reserved for structures that don't primarily aim every second of every day to "exit" from their own construction).. I suspect the remaining 20% "serious" puts that aspect in the right adrenaline-driven ultra-effective no-BS 'hungry hunting mode'/context and would remain crucial so that "play" doesn't drift off into 'massaging sandcastley no-one ever asked for'..

Reply on news.ycombinator.com