Hacker News Reader

And the premise that there can be reusability of software in that scale is the flow.

> Most devs are rebuilding identical software stacks to serve a very small different business layer product.

The problem is plainly because we don't recognize that the "Identical Business layer" does not exist. On the surface they seem identical but then each has its own features/prerequisites/peculiarities etc. And you end up with having to fight that "reusable framework" to handle all this because to be reusable it has to stay generic enough and the more generic something is the more useless it is for the concreate problem and more time consuming it is to reuse it.

In the golden era of Web Services the moto was to make reusable software same as electricity is reusable through standards (just plug in your device) forgetting that Electricity is reusable only because it doesn't really do anything. All implementation of the useful work is in the device you plug. And what's more these devices now are forced to work on non efficient standards, Hey I need just 2W of power to work but to interface with the Reusable thing I must waste 5w power to conversion. Fortunately we still don't have that kind of reusability in Software and hope we will never have to.

at the large scale sure. at the small scale no. For example, look at all the different implementations of JSONPath in all the various language that do exactly the same thing: take in some JSON, extract some data from a specific point in it based on a string input.

do we _really_ need to rewrite that every time we start a project in a different language that doesn't already have an implementation?

Or, could it be loaded/used via a service or other resource that's made accessible in some language agnostic way?

I work for a company that build embedded systems, commonly on a core family of processors. The way we get around this is to have an ever-expanding core set of reusable software that can be customized for each project. At the beginning of a project, you fork that core and add it to your project. If you identify a bug in the core, then you discuss it with the maintainers, do a PR and the fix gets merged back for everyone to use. Similarly for extensions that are likely to be usable across the company. Stuff that's only relevant to your project stays on your fork.

It's not perfect, but it saves an amazing amount of duplicated work. The problem is getting to be that the framework is getting big enough that people new to the company don't always know what's in it, and accidentally end up implementing something that already existed. We're still working on a solution to this.

The problem is getting to be that the framework is getting big enough that people new to the company don't always know what's in it, and accidentally end up implementing something that already existed. We're still working on a solution to this.

If you ever solve it, please let me know! Discoverability is hard.

Many years ago, we were trying to introduce a form of peer review process for the group I worked in, and we did a few exercises to see how well it worked in practice. I will always remember the striking result that around 50% of the issues raised during the first code review experiment were about reimplementing functionality that was already available somewhere else that the developer just hadn’t come across before.

I find this problem often happens with large standard libraries. Languages from Python to Haskell come with huge numbers of useful data structures and algorithms out of the box that can reduce a handful of lines of boilerplate to a one-liner with a standard name if you just use the provided tool. But of course, first you need to know that that tool exists, or at least have enough intuition that it might/should exist to check for it.

The best practical solution I’ve encountered so far is Haskell’s, where the strong culture around static typing means data structures and other important patterns tend to have explicit and well-known names, and where we have Hoogle, which is essentially a search engine that looks for known functions based on their input and output types. Need a function that checks if a given value is in a list but can’t remember what it’s called? Just search for `[a] -> a -> Bool` (Haskell syntax for a function taking a list of some type and a single value of that type and returning true/false) and you’ll soon find `elem`. Oh, and `notElem`. And their respective generalisations that work on types other than just lists, too.

In today’s world full of package managers and external dependencies and remote APIs and microeverythings, it feels like standardising this kind of search tool could have enormous benefits. It’s a problem that will always be difficult to automate away with tools given the fundamental limitations, but developers who know their own intent when writing code could certainly save a lot of time and make that code shorter and safer if we moved in this direction.

Spring cloud ecosystem is headed in this direction. Most of the common needs for micro service development such as Service mesh, service discovery, circuit breakers are getting incorporated into the ecosystem so as to reduce repetitive development. Not sure how many devs are actually using this.

They are building upon Netflix offerings.

https://spring.io/projects/spring-cloud-netflix

But then I look e.g. at your weather microservice, and see again a (configurable, but still expecting some already standardized format) wrapper around an existing API that once more mingles the data all around?

Sorry, but really not getting what this is achieving/solving?

Building a standard set of interfaces consumed as APIs regardless of what the backend implementation is. Something that exists on a network rather than as a library that can be executed and changed ad-hoc without having to upgrade thousands of binaries.

It also provides opportunities to do far more interesting things with the data when you have enough users at scale e.g predicting demand, caching data, etc.

What is the standard you built, how is it specced? In code that I just looked at?

Man I feel old and stupid, but really still not got it any further :( You made it even worse now.. "lives on the network.. without updating thousand binaries" - huh what, how?

You sound a bit like Urbit btw, not sure if appropriate ref though.

lol, really don't want to be in the same sentence as Urbit so please no.

Everything is defined as protobuf interfaces, which is a standard used by Google and everyone else now that gRPC is so dominant. So the idea is, define the API in protobuf, code generate and implement the handlers for it. The service can be called by other services on the platform using that code generation and then an API Gateway, which Micro provides can be used to call services externally using the same format but using HTTP/JSON.

To take that even further, M3O (m3o.com) codifies protobuf to openapi specs and then generates client libraries on top. You can see some of that in https://github.com/m3o/m3o.

Total tangent, but gRPC is not dominant and I think that belief is cargo-cultism.

Yeah I've managed to avoid gRPC for my whole career to date, and I've done a LOT of work on web services and microservices.

Thanks, that made now more sense. I'd put this condensed together with https://micro.dev/blog/2022/09/27/real-world-micro-services.... more prominently to the readme of https://github.com/micro/services ! Looking at that github alone makes it hard to commect the context.

Thanks I will do that!