Microservices [video]
youtube.com
youtube.com
One key part is that naming is important. This is always a fight I am willing to take. Acronyms, fantasy names, names of teams or legacy names that does not represent the functionality of the micro-service plague many companies.
If you will not name a method "Jimmy" or a local variable "Galactus", why developers are naming their micro-services in such a bad way?
I do not know what I find more frustrating, the bad naming per se. Or the fact that the same developers would lose their shit if they saw this:
class Galactus {
boolean Jimmy;
void turnOn() {
Jimmy = true;
}
void turnOff() {
Jimmy = false;
}
}
"Well, you see, the identity service isn't actually the source of truth for identities anymore ever since the oauth service started taking that responsibility and the plan was to move all the endpoints into the other service but some other priorities came up, but that was five years ago..."
That is, if agility of architectural development is one of the goals of microservices, names that don't get out of date because they never meant anything except "name for this codebase" in the first place might have some benefit.
Does that overrule the benefit of a common lingo you can talk about without the domain knowledge of your system? Probably heavily depends on way many variables than even HN can hash out, but there's an argument to be made here for these naming conventions at least.
> If you will not name a method "Jimmy" or a local variable "Galactus", why developers are naming their micro-services in such a bad way?
There was a post on the front page today about the "principle of least astonishment." It could not be more applicable than this example.
For those that missed it => https://wiki.c2.com/?PrincipleOfLeastAstonishment
This has been happening way before microservices became popular but it's gotten out of hand. Teams have nonsensical names as do their services. Outsiders coming in have a huge hill to climb to learn the domain because god knows why.
Please share any pointers you might have!
Their experience in my view is a representation of the architectural complexity and cognitive load you are inflicting with your design.
Similarly, we did a lot of learning from the tools we used that is probably very much just something you earn with battle scars. For example the behavior and issues we hit with our choice of core stack components (celery, redis, rabbitmq, kubernetes,...) are often things you learn by using them and gaining experience.
The next thing is there were a lot of conscious tradeoffs that I would still argue we would take today. We prioritized speed of building features to make the product viable and attractive to customers at the expense of quality of those features and reliability. Doing the hard real world testing instead of relying on simple unit/integration tests before shipping. That's something we now are having to address but the question is was that something we should have done differently? No sure I would, we may not be around now if we hadn't built the feature we needed to sell the product and prove market viability.
On the topic of this video. I think the issue it not using microservices or building a monolith but controlling complexity. The biggest thing I dislike about our current infrastructure is the cognitive load and complexity that a new engineer joining has to deal with to be productive. That can happen in any architecture if you aren't careful. Sure having to run multiple services, tracing and testing acrross them (basically the usual gripes about microservice architectures) dealing with inter-service calls, performance tradeoff etc.. is not as good as I'd like it to be but it could be improved. What sucks most is there a lot of raw edges we left exposed that trip new people and that I think was a mistake we should be prioritizing fixing those as they come up and think of them while buidling up our core framework and components more thoroughly.
The Nike.com software group was over 1,000 employees working on a microservices mess that performed horribly on the customer side. They could have built a much better product with about 50 competent software professionals.
It oversimplifies things a bit, though.
It's odd; I've never seen a "monolith" simplified by moving to microservices, but everyone seems to be doing them.
From a high-level perspective, is there really any difference between DMI and HTTP/gRPC/et.al. if you could hypothetically run all of your services on a single logical node? If your entire production stack runs on your macbook, why cant you just wire it all up as a single binary and throw it on a nice powerful server? I guarantee it will run better without all that ridiculous network glue in the middle.
If I am being modest, I would say that we feel really, really stupid from where we are sitting right now. Primarily for having previously spread this implementation across 10+ services talking HTTP for absolutely no good reason. I am growing into a mindset where the entire business system needs to fit into 1 binary image. Perhaps duplicated across more than 1 physical host with differing features enabled per, but at least sourced from the same solution/repository.
What microservices allow is for different teams to use different technologies and to upgrade components independently. On service might by Python+postgresql, another Ruby, and third also Ruby but two versions ahead.
It feels fast and agile in the short-term to decouple teams like this. In the long-term, it's almost always a maintenance nightmare. You end up with way too many different technologies.
I usually break up my projects into libraries (in Python, independent pip-installable packages) with their own unit tests, and some pretty weak glue in the middle. The service layer is omitted. I design things to scale horizontally too.
I've built apps with a few services, but I've never seen a good example of a microservice architecture that beat good old-fashioned clean abstractions in libraries.
Every few years, someone comes up with a new way to do abstraction, and compares that to no abstraction (rather than good abstraction with the previous model. Microservices seems like the dumbest of the bunch I've run across in my career.
If the microservice architecture has made you less productive than the top leadership is to be blamed.
Microservices are all about scaling development agility in exchange for increased technical overhead.
In many cases this tradeoff is atrocious. If you can say "we can easily take on (large feature XXX) and deliver that", microservices are all pain and no gain.
Where they really start feeling good is when your "two week sprints" turn into about two days of available dev work because the remaining time is spent regression testing your changes and trying to keep the build from breaking because of tests you didn't touch failing (spooky action at a distance is real and it hurts). A clear API contract that both sides of the fence can develop against means that you can keep making progress.
At least until it's time to integrate against the real world, not your mock ;-)
Microservices can be a great pattern to deliver quickly _and_ maintain high availability/low technical overhead, that's their point. I think the problem arises when developers try to over-optimize or over-engineer a system, or they want to get promoted so they write a new service, ending up with huge entangled webs like in the sketch.
Individuals and interactions over processes and tools, every time
Yes – I was talking about those same monolithic architectures in that comment ;-)
As you move up in leadership, the tool that you have is organizational design. Doing it right and you have a highly efficient org. Do it wrong, and you'll constantly hit roadblocks.
How's that working out for you?
In most cases, and I'm talking right up to the scale of very large enterprises[1] you don't need that much complexity to scale technically and, if you avoid the complexity, you don't need that many people either.
And I don't think you can ever afford to be myopic if you want to effectively deliver solutions to real business problems.
[1] 90k employees, specifically, but also, specifically, not a tech company. A traditional company with a tech function.
The latest system I designed professionally was an ordering system that's divided into two parts; a local application with a database and an HTML document and a PHP script on a server.
The application itself doesn't need internet access to work.
The data needed for the online customer order module is uploaded to the server via FTP as needed. Online orders are stored on the server and polled via FTP on request from the local application.
The result is modular, very easy to maintain, and performance is good enough to be considered a feature.
But the reasons for the choices I made are all about the problem being solved, not organizing developers and keeping them from stepping on each others toes.
Anytime something is discussed, we ask ourselves if that change will improve users' lives. I am sure these are useful for other people at different maturity levels of software, but it is not for us, right now.
This is why we don't use React, Angural, Webpack and a ton of other things and libraries that are useful for others but add no value to us and only increase. We avoid adding complexity unless it's really worth it, and worth it for the users, not to give us pleasure of using a new library. We have side projects for that.
1) There is a codepath in the application that has radically different load characteristics.
2) We want to give each team in the organisation ownership of (most of) their stack, release cadence, SRE etc.
(2) seems like an attempt at engineering but is often about management - empowering the teams yada yada.
There is a thread here about: "Technology opinion-sphere is a FAANG monopoly" aka "Dude, we got like 3 paying customers!". I'll leave that for another day.
I feel like there is no accountability for these clowns. They'll inevitably switch jobs and ruin the work lives of so many developers with their stupidity. There is a culture of not wanting to name your abuser for some reason. It's weird.
At a previous employer I remember they had someone build a React front-end which talks to a GraphQL proxy which itself talks to a REST-based API and seemed proud of his creation. There was absolutely no reason why the front-end couldn't talk to the REST-based API directly (it was on the same domain, and the front-end already had a valid auth token), but now there's one extra moving part in the system that can (and did) fail.
At another place they for some reason decided that they needed React despite only a single page of the application actually needed to be "real time". 9 months later they still didn't even have a functional registration form and then it took them over a day for the basic feature of displaying the QR code for two-factor auth enrolment. The same signup form would take me less than a day to do in its entirety using "boring" HTML pages and forms (which our framework already had lots of helpers for, so the backend code for it would've been around 100 lines).
My manager want's "pretty URL's" even though it's not a good idea to bookmark CRUD detail pages. It creates internal code messes to keep them pretty. It's yet another me-too feature that bloats the stack. I proposed a compromise, but he really wants them pretty.
There are too few K.I.S.S Cops in IT; the Bloat Mafia reigns. Warren Buffett says part of the secret to getting rich is confidence and conviction to say "no" when the logic and numbers don't support an idea but peer pressure says "yes". https://news.ycombinator.com/item?id=22795294
No one wants to burn bridges. The world gets really small at the most inconvenient times.
You really have to work within the system, and in the end, become the same system.
Imagine playing competitive RTS game and decisions were made with this attitude, tomorrow-not-my-problem as others don't have domain expertise to tell otherwise or they don't simply care to put themselves as opposing force and do your work to find better solution. In game you'd be doing seemingly ok and then get destroyed all of the sudden.
(Edit: at the casual level)
Is there an equivalent of "cyclomatic complexity"(McCabe count of code) for cloud/distributed systems/μServices architecture? Cant find any such objective measure, apart from a few dated research papers.
IMO, it could be useful for many of us. Sometime back, I had tweeted tagging @copyconstruct and @kelseyhightower asking this question, but haven't got a response - may be it went unnoticed.
Would love to get to know if there's any such method/framework to measure and manage architectural complexity, for the world in cloud.
There are pros and cons to each approach, and the details matter with regard to how services will be scaled and the coordination of that scaling happens.
There are lots of finer points in the details, but knowing the difference between the two approaches can help with the decision making process on which path the adopt.