Hacker News Reader

Comment by wvenable | Hacker News Reader

As a programmer, if you don't create an abstraction you'll never be more than a 1x programmer. Abstraction is how to get more productive than simply how fast you can type.

Yes, the wrong abstraction is bad. But almost every argument whether it's for/against duplication or for/against abstraction usually starts with the hidden premise that you're stuck with whatever choice you've made and code you've written forever. The underlying issue is the fear of change and the sunk cost fallacy of already written code. If you have the wrong abstraction, you can change it. If you created too much duplication, you can remove it.

View on HN

> The underlying issue is the fear of change and the sunk cost fallacy of already written code. If you have the wrong abstraction, you can change it.

It's not that trivial. Consider that the wrong abstraction is reflected into your API (common), and consider that your API has many users. You are stuck with it, or you have to convince multiple teams (or, god forbid, external customers) to migrate to a better API. This can constitute a humongous waste of SWE-hours ($$$$$) and take quarters to accomplish, assuming you can get any buy in.

I think it comes down to what your organization looks like and how many users are going to be touching your code. If your abstraction is just for yourself internally and everyone else is not allowed to touch it, then fine. You will own the tech debt if the abstraction is wrong. If your abstraction has other users at your company, or external customers, it had better be the right one or at least an unavoidable stepping stone.

> If you created too much duplication, you can remove it.

This is actually true. Refactoring duplicated logic is a lot easier than fixing bad abstractions.

It is that trivial. There is no alternative. You either have an API or you don't and you either change it or you don't. Hand wringing over the potential of making a mistake is a waste of time and effort. You will make a mistake. You will never get it perfect. You just have to deal with it.

> Refactoring duplicated logic is a lot easier than fixing bad abstractions.

Then you've just created an abstraction with all that potential to be bad sometime in the future.

> It is that trivial.

> You will make a mistake. You will never get it perfect. You just have to deal with it.

These two comments sound at odds. First statement says it's easy. Second statement says it's hard.

We can agree that hard things don't get solved without iterating. But a productive response to abstraction (which is really API design) being hard is not to say "stop handwringing, just do it." Instead, you can employ various strategies such as preferring experienced people to do it, making sure they did a good job of gathering requirements and considered the risks of their approach, spending time testing customer/developer ergonomics, etc. You can also defer producing an abstraction until your system is a bit larger and the duplication is becoming too much to handle, since you have a larger sample size of potential uses for your abstraction to help you converge on the correct API.

Good abstractions can be the difference between success and failure, between organizational velocity and technical debt quagmire. Saying "we should always build abstractions" when it's difficult to build them correctly in one go sounds totally wrong on its face.

Trivial doesn't mean easy; it's means unimportant. And in this case I'm referring to the whole idea of abstraction or not. You're going to do it. You should do it. Sometimes you have to do it. Discussing it is pointless. Just move on to the how.

> Saying "we should always build abstractions" when it's difficult to build them correctly in one go sounds totally wrong on its face.

If you don't do that first "one go" how do get around to right abstraction? You're telling me you wouldn't use any intuition, planning, or thinking to build that first wrong abstraction? It seems above you just described how to do exactly that. Just writing without any abstraction in mind at all is just a huge waste of time. The only time it's painful is when you decide that you can't change anything you've done. But doing no abstraction is just as painful -- you just pay for it differently.

Okay Mr. English, trivial may be defined as unimportant / trifling but if you're gonna label something like that then you have you consider why you're calling it unimportant. Still seems you're saying it's cheap, easy, and undoable. It's not.

Abstraction building is _consequential_.

> If you don't do that first "one go" how do get around to right abstraction?

Right back at you: how do you get the data about how to build your abstraction without first living without one to understand what parts of the system should be abstracted? Starting with the abstraction and assuming you can fix it later is a luxury for overstaffed teams and people working on systems that nobody actually uses.

You do realize, abstractions incur mental load on your users? Have you ever tried to use internal tools built ostensibly for your use case, only to find that it's taking more time and effort than if you lived without one? It's happened to me so many times, and the consequence of adopting a shitty abstraction has burned me so many times personally that I'm gravely aware of the downsides.

Consider a streaming data tool that stupidly allowed users to put arbitrary data in a string field, which became abused for purposes it shouldn't have been used for -- now it's critical to the company and would take a YEAR of dev resources to retire, all the while being an endless source of headache and P0 SRE escalations.

Consider an ML evaluation tool that requires you to understand how a bunch of leaky abstractions nest together. When you finally figure out how it works and it does 1 thing for you (generate P/R curves), it becomes extremely difficult to modify or debug without again understanding these horrible abstractions. Once you've spent a full quarter adopting it, you wish your team had just written something custom-but-maintainble instead of trying to get shoehorned into a tool that clearly wasn't thinking about you, and whose owners have left the company leaving you to absorb the cost.

> But doing no abstraction is just as painful -- you just pay for it differently.

It's less painful in the long run: doing no abstraction to start is often the right path. People often overrate the benefits of abstraction and underrate the clarity of repetition, especially when the repetition is trivial.

> how do you get the data about how to build your abstraction without first living without one to understand what parts of the system should be abstracted?

The minute I have any duplicated logic it gets combined as much as possible. I hate duplicating anything. Some duplication is impossible to avoid and I hate that too. I hate specifying the same logic in 2 places (say client-side and server-side for web validation). I will do anything possible to avoid that.

Good abstractions reduce mental load. I wouldn't be able to manage 20+ different applications if they didn't share a huge amount of common internal framework and have as a little duplication of logic as possible. I'm actually pretty militant about ensuring every abstraction is a programmer benefit and will remove layers if they serve no purpose.

I almost always build things out as a library. If we are consuming an external service, I will make that it's own library/framework to be consumed by the project it interacts with. Even if it's only ever used by one project. This is always more work initially but has never failed to be right choice.

> Good abstractions reduce mental load.

Yes, in sum, but bad ones increase it, and it's not always immediately obvious to everyone that an abstraction is bad. A lot of that comes from experience.

> The minute I have any duplicated logic it gets combined as much as possible. I hate duplicating anything. Some duplication is impossible to avoid and I hate that too. I hate specifying the same logic in 2 places (say client-side and server-side for web validation). I will do anything possible to avoid that.

Good for you Glenn Coco! Work in any scaled software organization and you'll see that combining logic too early is full of hazard.

> I almost always build things out as a library ... This is always more work initially but has never failed to be right choice.

Maybe to you. I wonder if someone who's ever tried to use your library down the line ever thought "this was abstracted in a really suboptimal way but I have to live with it." Even if they did, I don't think you'd know. I've been that person too many times. People have got to stop building shitty libraries when they're not necessary and only serve to obfuscate the logic.

You argument just boils down to bad programmers writing bad code. It has nothing to do with abstraction or not. You'd be just as unhappy with an entire project that's just one grant 20,000 line file.

That's why I think the whole rant about abstraction is pointing fingers in the wrong place. Everyone either wants to find the silver bullet that will produce code perfectly the first time or find some concept to blame when it isn't. If you have experience, use abstraction to accelerate your development.

> Work in any scaled software organization and you'll see that combining logic too early is full of hazard.

Don't combine logic and you need twice as many programmers to do the work. That front-end guy to do the JavaScript for the validation, the backend guy to do the server validation, the project manager to ensure they're both always the same. I started this rant by saying if you don't abstract, you'll always be slow. And I stand by that; many projects with dozens of programmers are just manually performing work that could be abstracted away at the start.

> You argument just boils down to bad programmers writing bad code.

Good programmers can write bad abstractions as well, because good abstractions require sufficient understanding of what you're trying to abstract and we have blind spots (the unknown unknown). Thus the caution.

> It has nothing to do with abstraction or not.

"Bad code" is one thing, "bad abstraction" is a very specific subset of that problem.

> Everyone either wants to find the silver bullet that will produce code perfectly the first time or find some concept to blame when it isn't.

I think you're missing the point. Bad abstractions, especially when depended on by many users, are 10x bugs. Nobody's saying "make all code procedural and never abstract anything" -- however it's very valid to say "problems caused by bad abstractions are super bad so let's be extra careful." I mentioned strategies for being extra careful earlier: let more experienced folks do the design, defer the abstraction until you have more information, etc.

There's also levels to everything. Of course there's no repercussions for how you structured the classes in your internal facing webapp. Nobody cares about your codebase except you. If you're building a foundational building block of a complex system however (e.g. the message bus for a self driving car) you had better make the best approximation of the right answer from the get go, because that system isn't going to be re-built for many years.

> Don't combine logic and you need twice as many programmers to do the work.

This doesn't ring true. Duplicated logic doesn't always mean double work.

> I started this rant by saying if you don't abstract, you'll always be slow.

Yes and my point is: if you think abstracting is always a net win, you're probably green and haven't seen the myriad cases where it bites you.

I'm anything but green. And I have, of course, made all these mistakes over my career. But if you're capable of running in 5th gear, don't take advice that says stay in 1st until you've finished the product.

> Then you've just created an abstraction with all that potential to be bad sometime in the future.

I would argue that you've then created an abstraction, but with all the hindsight allowing you to create the _correct_ abstraction (or at least a much better chance at it approaching "correct")

100% agree, especially about refactoring duplicated logic. Super-duplicated code begs to be refactored, and having many examples of the same functionality helps you build an API that is robust without adding "what-if" functionality to try to futureproof code (impossible).

Migrating to better APIs is done all the time. It is not an issue worth discussing about anymore. But even if you have to maintain an API this doesn't mean you cannot change the underlying implementation.

It absolutely is an issue worth discussing, any time you are maintaining a library with more than a few other people using it.

Breaking backwards compatibility in a library with hundreds or thousands of users is not something to be taken lightly!

I'm working with an API right now that is absolutely based on duplicated code. They have a system for querying items and the API has 3 different ways depending on the endpoint. I just found a new one the other day and I hated it -- why is this one API unnecessarily different from all the rest!

I'm building a library to call this API and I've abstracted over all these differences so my callers never have to know how messed up the underlying API is -- they get a consistent experience regardless.

I'm not sure that's true, when I look at the code bases of some of the most productive coders I can think of (John Carmack and the Doom and Quake code bases for example), they tend to be fairly conservative with how much they abstract things. There's a lot of very thoughtful data structure usage (git is another good example of this) and diligence in maintaining a coding standard (which generally has little to do with formatting), but most of the code seems to be more concrete and task focused rather than abstract.

I think Casey Muratori has a good way of thinking about this, with his concept of "semantic compression" ( https://caseymuratori.com/blog_0015 ). To me that's a lot more valuable than the ideas that you get in say something like Clean Code (which in my opinion, the popularity of that book has been a disaster for the industry)

I'm positive they don't have much in the way of duplicated code.

Abstraction can be as simple as a function.

It seems odd to give really good examples of abstraction and then sort of do a No true Scotsman argument on it. "It can't be abstraction because it's not terrible."

so your argument is, "you've got to write functions, therefore abstraction is always good" ?

yeah man i don't start in main() and never write another function ever again. You got me. But I also don't aggressively police duplication, accepting that I would always rather see what's happening without file-hopping. The correct abstractions will simplify code, and they will seem obvious (even if just in retrospect). Abstractions that force me to continually re-frame the problem i am trying to solve in terms of another's use case are antithetical to writing the sort of code that I do.

You don't have to write functions. Have you ever seen a 10,000 line code that was just a single function -- I have. It also had nested if statements 5 levels deep to handle all sorts of logic with lots of duplication. It was unmaintainable. Yet it did meaningful work and it could have been refactored to a fraction of the size and do the same work. But, to be honest, when I had to fix it I just went it and fixed in the 20 places that needed changing because it was impossible to follow.

You like good abstractions and you hate bad abstractions. I couldn't agree with that more.

There are useful abstractions and useless abstractions. A lot of the GoF designs are bad abstractions (if used indiscriminately) and clutches for bad languages. However, using problem-focused abstractions is a big time saving strategy.

I’m sorry you are totally, utterly wrong.

Wrong abstractions percolates to your system, assumptions on how your abstraction is supposed to work is hard ossification that hides concrete implementation and their actual, generally simpler, contraints.

Basically your refactoring work now requires to understand all user code that relies on the wrong aspects of your abstractions find a way to correct it if your lucky and make it work exactly the same way the duplicated code would have.

And I didn’t mention implementations that drifts in incompatibles ways with the abstraction, a large source of errors and regrets.

The good bet for productivity is recognizable implementations patterns and duplication.

In the end refactoring duplicated code that had time to settle and drifted in legitimate ways to find your correct abstraction is a blast.

I disagree and I can provide an example. I'm creating a library to interface with a REST API. The creators of this REST API obviously didn't do any abstractions and they have multiple implementations for the same exact process: paged queries of items. There's no reason for them to be different -- they're just different because, I assume, different developers built them differently at different times. Nobody looked at this and said "This is all the same so we should have one single common implementation abstracting over all the endpoints."

However, as the developer of the interface library, I can abstract over all the differences and give my consumers the same exact same experience regardless of the API endpoint. And that's exactly what I did. So now they're all more productive because they don't need to know all these unimportant details. They don't even need to know that there's REST API. In fact, this this REST API replaces a previous API implemented with a completely different technology and we are swapping the whole thing out with minimal changes because I abstracted it years ago.

Not all abstractions are wrong. Not all concrete implementations are simpler. My goal with an abstraction is to take some else and make closer to what we need because most technology has a wide audience with wide requirements. I'm a narrow audience with narrow requirements and so I can hide the vast complexity that I simply don't care about.

Wrong abstractions start off as right abstractions and slowly become wrong abstractions. What's the point in which a right abstractions becomes a wrong abstraction? Am I sure I can identify that point? Can someone else? Can someone else which has no knowledge of the original assumptions that were implied during the initial abstraction?

There are two kinds of abstractions in your, the ones everyone complains about and the ones that no one has ever seen.

My rule of thumb is thus: Have I repeated myself three times doing the EXACT same thing? Then CONSIDER abstracting away. Otherwise, make as many implicit dependencies explicit as possible and slightly keep repeating yourself until you are exactly repeating yourself

Your rule of thumb isn't great: If you have some important logic duplicated twice but a year from now has a bug -- are you going to remember to change it both places? But, lets be honest, you probably would not create that code in the first place -- you'd have abstracted it automatically without even thinking about it.

These conversations generally tend to completely discount experience. Junior programs are often terrible at abstractions -- they either do way too much or way too little. Can I give them a hard and fast rule that they can use to never make that mistake? No, I can't. It doesn't exist. The only reason I know what's good or bad is because I've done it wrong thousands of times.

That's the problem with every single one of these articles that prescribe one true solution. It's not at all that simple.

Reply on news.ycombinator.com