And yet we do this all the time. Your CI/CD blocking your PRs until tests pass? That's a costly technical solution to solve an issue of organizational discipline.
But architectural complexity has outsized long-term cost, and service-orientation in particular has a _lot_ of it. And in this particular case, it doesn't actually solve the problem, since you _can't_ successfully enforce those domain boundaries unless you already have them well-defined.
> But architectural complexity has outsized long-term cost
As do technical solutions, of course. CI/CD systems are very expensive, just from a monetary perspective, but also impose significant burdens to developers in terms of blocking PRs, especially if there are flaky or expensive tests.
> And in this particular case, it doesn't actually solve the problem, since you _can't_ successfully enforce those domain boundaries unless you already have them well-defined.
Ignoring microservices, just focusing on underlying SoA for a moment, the boundary is the process. That is an enforceable boundary. I think what you're saying amounts to, in microservice parlance, that there is no way to prevent a single microservice from crossing multiple bounded contexts, that it ultimately relies on developers. This is true, but it's also just as true for good monolithic designs around modules - there is no technical constraint for a module to not expand into domains, becoming cluttered and overly complex.
Microservices do not make that problem harder, but SoA does give you a powerful technical tool for isolation.
Not concisely in the general case, but in this case the difference is fairly straightforward - CI/CD doesn't affect the structure of your executing application at all, only the surrounding context. I don't want to spend the hours it would take to characterize architecture as distinct from implementation, but the vast number of textbooks on the topic all generally agree that there is one, though they draw the lines in slightly different places.
> I think what you're saying amounts to,
Very much no - my point is about the process of implementation. The services.. _do_ enforce boundaries, but the boundaries they enforce may not be good ones.
In order to successfully extract services from a monolith, you have to go through a process that includes finding and creating those domain boundaries for the domain being extracted. If it's your first time, you might be doing that implicitly and without realizing it's what you're doing, but under the hood it's the bulk of the mental effort.
The part where you actually _introduce a service_ can be anywhere from a tenth to half of the work (that fraction varies a lot by technical stack and depending on how coupled the domain in question is to the central behavioral tangle in the monolith), but by the time you've gotten the domain boundary created you've _already solved_ the original problem. Now you're facing a trade of "extract this well-factored domain out to a separate service application, to prevent its now-clear boundaries from being violated in the future". And I contend that that's a trade that should rarely be made.
The technical part of services is easy enough really but if the organisation leaks its boundaries, which always flow downstream into the actual teams, you are proper fucked.
It takes a different level of maturity, both in organisation and team, to build a service oriented software.
That's my experience at least.
People are not reliable enough to leave them as the only protection against system failure...
I don't think that rule can be applied as universally as you stated it. But then, I have never seen anybody breaking it in a bad way that did also break it in a good way, so the people that need to hear it will have no problem with the simplified version until they grow a bit.
Anyway, that problem is very general of software development methods. Almost every one of them is contextual. And people start without the maturity to discern the context from the advice, so they tend to overgeneralize what they see.
Because the complexity of that outer system is also important, but there are a few very major differences between the two that are probably too obvious to belabor. But in general, architectural complexity in the inner system costs a lot more than it does in the outer system, because it's both higher churn (development practices change much slower than most products) and higher risk (taking production systems offline is much less permissible than freezing deployments)
No, I'm not. Are you overlooking some of the impacts of your tests and most of the impact of static verification?
Those do absolutely impact your system, not only your environment. For tests it's good to keep those impacts at a minimum (for static verification you want to maximize them), but they still have some.
I'm not sure about the tests - it depends on what you'd consider an impact possibly; I've been trained by my context to design code to be decomposable in a way that _is_ easily testable, but I'm not sure that is really an 'impact of the tests' (and I'm fairly confident that it makes the abstractions less complex instead of more).
Would you mind explaining what you mean in more detail?
Like any good test system I have a large suite of "unit" tests that run quick that I run locally before committing code - it takes a few minutes to get high code coverage if you care about that metric. Even then I just run the tests for x86-64, if they fail on arm that is something for my CI system to figure out for me.
If you got _reamed_ you got off lucky: in plenty of places you'd be out on the street.
It may well be that you had it right but from context as given I hope this shows you some alternative perspective that might give you pause the next time you decide to throw out the rulebook, even in emergencies - especially in emergencies - these rules are there to keep you, your team and the company safe. In regulated industries you can multiply all of that by a factor of five or so.
But not being in a regulated industry doesn't mean there isn't a very good reason to have a code review in your process, assuming it is used effectively and not for nitpicking.
Not having a code review step is usually a bad idea, unless everybody on your team is of absolutely amazing quality and they never make silly mistakes. I've yet to come across a team like that, but maybe you are the exception to the rule.
Why? Because it's a "good practice"? They have 12 people and no customers, they can almost certainly adopt a very aggressive developer cycle that optimizes almost exclusively for happy-path velocity. You'd never do that at 50+ engineers with customers but for 12 engineers who have no customers? It's fine, in fact it's ideal.
> with massive downtime as a result.
They have no customers, downtime literally does not exist for them. You are following a dogmatic practice that is optimizing for a situation that literally does not exist within their company.
If you're wondering why software is crap: it is because every new generation of coders insists on making all the same mistakes all over again. Learn from the past, understand that 'good practice' has been established over many years of very expensive mistakes. 12 engineers is already a nice little recipe for pulling in 12 directions at once and even if they're all perfect they can still learn from looking at each others code and it will ensure that there are no critical dependencies on single individuals (which can bite you hard if one of them decides to leave, not unheard of in a startup) and that if need be labor can be re-divided without too much hassle.
Dogmatic adherence to a process that limits developer velocity and optimizes for correct code is very likely the wrong call when you have no customers.
I don't know the context and I don't know the particular business the OP is talking about. What I do know is that if you feel that your management is cargo culting development methodology (which really does happen) you can either engage them constructively or you can leave for a better company. Going in with a confrontational mindset isn't going to be a good experience for anybody involved. Case in point: the OP is still upset enough that he feels it necessary to vent about this in an online forum.
Note that this is the same person who in another comment wrote:
"On the flip side I’m trying to convince my CTO to fire half our engineering team - a group of jokers he hired during the run-up who are now wildly overpaid and massively under-delivering. With all the tech talent out there I’m convinced we’d replace them all within a week."
> Case in point: the OP is still upset enough that he feels it necessary to vent about this in an online forum.
I apologize for the conversation starter.
Your attitude towards code-review on the other hand is one I've seen before several times, and I was glad when each of those people were fired.
If we waited for reviews before merging, we’d be waiting weeks to merge a single PR. Thus, you wrote your code, opened a PR, did a self-review, then deployed it. We had millions of customers, downtime was a real possibility. So you’d watch metrics and revert if anything looked slightly off.
You would wake up to your PR being reviewed. Sometimes there would be mistakes pointed out, suggestions to improve it, etc. Sometimes it was just a thumbs up emoji.
The point is, there are many ways to skin this cat and to “ream” someone for merging without deploying is incredibly immature and uncreative. You can still review a merged PR.
> to “ream” someone for merging without deploying is incredibly immature and uncreative.
I'd agree, but I _highly_ doubt that description was an accurate one. Read through the other comments by the same person and you'll get a picture of their personality pretty quickly.
It's likely that there was already an ongoing conflict either in general or specifically between them about this issue. They probably got a moderately harsh comment to the effect of "hey, you're expected to wait for code-reviews now, knock it off"
You want to get the team to a point where you can disagree and commit, no code will ever be perfect and there is no reason spending 3-4 rounds of change requests trying. I think the worst code review I ever had, ended with me saying, “if you’re going to be this nitpicky, why don’t you take the ticket?” (It was extremely complex and hard to read — and there wasn’t any getting around it, lots of math, bit shifting, and other shenanigans. The reviewer kept making suggestions that would result in bugs, and then make more suggestions…)
He came back the next day and approved my PR once he understood the problem I was trying to solve.
Even these days, where I work on a close team IRL, I’ve been known to say, “if there are no objections, I’m merging this unreviewed code.” And then I usually get a thumbs up from the team, or they say something like “oh, I wanted to take a look at that. Give me a few mins I got sidetracked!” And I’ve even heard, “I already reviewed it, I just forgot to push approve!”
Communication is key in a team. Often, if the team is taking a long time to review, give them the benefit of the doubt, but don’t let yourself get blocked by a review.
Anything else is un-needed grooming that's more about the other developer's ego, not about good code (sometimes its to follow some other constraint, but its a good sign the person has a personality issue).
> They have 12 people and no customers, they can almost certainly adopt a very aggressive developer cycle that optimizes almost exclusively for happy-path velocity. You'd never do that at 50+ engineers with customers but for 12 engineers who have no customers? It's fine, in fact it's ideal.
12 engineers churning out code with no code-review at all? That'll produce velocity, for sure. It'll also produce not just an unmaintainable mess, but an interesting experiment, in which you get to find out which of your engineers are socially capable enough to initiate technical communication independently and construct technical rapport _without_ that process helping them to do so. Hope none of them hold strong technical opinions that clash!
During all this time, I've seen plenty of "small changes" having completely unexpected consequences, and sometimes all it would take to avoid would someone else seeing it from another perspective.
I’m not convinced bad code would get merged more often if we didn’t require approvals. I am convinced we’d deliver code faster though, and that’s what I’m trying to optimize for. Your company and engineering problems are not the same as mine.
I agree with everything you say here, but honestly it’s quite ineffective. I wish we had found something better than unit tests (often misleading and fragile) and the ability of a qualified reviewer to maintain attention and context that a real review entails.
The primary value I've seen across teams has been more on having shared team context across a codebase, if something goes bump in the night you've got a sense on how that part of the codebase works. It's also a great opportunity for other engineers to ask "why" and explain parts that aren't obvious or other context that's relevant. We'll find the occasional architectural mismatch(although we like to catch those much earlier in the design process) and certainly prevented bugs from shipping but if that's the primary focus I think a team is missing a lot of the value from regular code reviews.
At the low end of engineering, yeah, code reviews matter a ton and do catch bugs even if they’re basically just peephole inspections.