If you think home-rolling a solution is the answer | Hacker News Reader

If you think home-rolling a solution is the answer

twitter.com

10 pointsmooreds23 comments

This is really reminding me of a cargo cult. I observe that there is a prevailing mindset among software engineers that their peers are always better than them, and if there is a "standard" solution, it will always be better, more performant, easier to use, bugfree, etc. compared to a quick home-rolled version.

This is a good thing if you want to contribute to software bloat, but most use-cases can be solved by rolling your own, then adding features as needed.

I have a specific example - I wanted to do logging in a program I made. I experimented with adding different loggers to the project but all of them were more complex and harder to set up than what was needed. Ultimately, I've written a logger which does exactly what I need and in the exact same way in a few hours. (the standard loggers needed some finicking to produce output in the format I wanted)

> it will always be better, more performant, easier to use, bugfree, etc. compared to a quick home-rolled version.

You're leaving out crucial time factors:

* which is quicker to market

* which has the lower maintenance burden

Buying, in my experience, wins on those axes.

It does, though, depend on the size of the problem, which is a good callout. You rolled your own logger for a known format. You didn't roll your own RDBMS.

Also, you experimented with other solutions to determine if they were feasible. That's exactly the kind of experimentation we should all do--survey the problem space before jumping into building something.

I daresay you prove the tweet's point. For your problem, your solution was way better than the other options.

You are right in the fact that for complex systems, it will always be quicker to use an off-the-shelf solution than making your own. However, for maintainability, the most important metric is the raw amount of code, followed by the engineers' abilities and the code's level of documentedness.

So, if you can replicate what you need with a third of the code, you are already in a much better position (and can fix your own code quicker than a third party's!) than if you had to wait for the vendor/a poor guy reviewing your pull request.

A standard version, when used the same way as everyone else, has already been tested. You might run into bugs, but I don't see as many as with new things.

Performance often doesn't matter that much, because your program often only spends a trivial amount of CPU doing something. I don't really care about a few microseconds to determine a file type given a name, when there's 10x longer to decompress the file and 100x longer to even load the dialog to find it.

For the things that the program actually does spend a lot of time on, there's usually a way to optimize it with hardware instructions, GPU stuff, special case algorithmic shortcuts, etc, which are hard and tedious to do for multiple platforms. The ultracomplex solution might be just as fast.

Plus, DIY stuff is rarely general enough to be reusable. The standard solution might be hard to set up, but it will probably be less work that writing something from scratch.

And it will almost certainly be less work for a maintainer to understand. They probably won't have to learn anything at all, they've seen all the patterns before on every project.

DIY is great if you value simplicity just for it's own sake, otherwise I'm not sure I see the appeal.

A lot of programmers seem to get into programming specifically because they value simplicity, they like to understand and work with new ideas and complexity gets in the way of understanding, so I see why they might like to about reuse.

"Performance often doesn't matter that much, because your program often only spends a trivial amount of CPU doing something. I don't really care about a few microseconds to determine a file type given a name, when there's 10x longer to decompress the file and 100x longer to even load the dialog to find it."

I agree with the rest but this is how we are at a point where we have to wait for windows to open. Unlike on Windows NT. It's not a problem of one thing being slow - if everything is being slightly slow, it's a death by thousand cuts.

I think most of the problem with waiting for windows to open is a matter of lack of caching, or waiting on some process that could have been parallel, or something like that.

Rarely do I ever see a library that takes more than milliseconds to import, or something slowed down on a function only called on a click or every few minutes or otherwise not in an inner loop.

I see lots of slow file open dialogs, which I'm told is often because of checking for network drives and things like that. I've seen exactly one library that was causing me so much performance trouble I had to ditch it, and that was an rrules engine.

Finding the next occurrence of something like "Every five minutes on Thursday" was taking an insane amount of time, but that's because I was calling it constantly to schedule lots of different things that happened very frequently.

I've also seen lots of things be slow because they wait for the network too. But basic stuff that libraries are normally used for generally seem to be fine. I don't think there's any way I could write a faster gaussian blur than Pillow or a faster text search than the default regex libs.

Most applications take seconds to open - and that's with fairly high-end hardware and an NVMe drive.

The worst offenders are the "webpages packaged as programs" (electron, CEF) and UWP.

I don't think the lack of caching is the problem - the real problem is doing so much work to open the application which requires caching.

Years ago, I home rolled my own web server.

Why? Security --- it is dedicated and limited to a single purpose so every request can be scrutinized and validated and verified. Attack surface is minimized and totally unique so any hacker is starting from ground zero. The only common vulnerability is the OS network stack.

After 25 years of being openly exposed and attacked on the internet, it has never been breached --- at least to my knowledge. In that time, I think I've saved a lot of time and money watching exploits for commonly used web servers just pass me by.

This is a good point (at a small scale until you become large enough to be a target people spend significant time targeting)

People target my server every single day. They mostly try off the shelf attacks geared toward known vulnerabilities and common misconfigurations of popular web servers and setups.

And these don't work --- so they mostly give up after a while.

Because those are more likely to yield results than trying to figure out how to break your custom web server. If a big company suddenly started using it or AWS made your custom web server a default option then that would be a different scenario.

So security by obscurity does work?

Until someone decides that your approach shouldn’t be obscure yes

It’s like the non standard locks that are easier to pick if you know how but can’t be picked with regular lock picking tools

Frosgren comes off like a newb, because they equate home-rolling with downtime and being embarrassed explaining to your customer what you do.

When I home-roll I have less downtime usually than relying on a vendor solution. But I have been doing this a very long time so maybe it is an unfair comparison; of course, I’m not so gauche that I am prefixing Dr in front of my name on Twitter.

> Frosgren seems like a newb

She co-authored Accelerate: https://www.amazon.com/Accelerate-Software-Performing-Techno... . She is also currently at Microsoft Research and was previously at Google and GitHub. This may be a hot take you disagree with, but she's no newbie.

I took her point to be "make sure anything you build is a lot better than anything you can buy, because there are a lot of unexpected expenses in building".

Co-author means they could have been carried. I have worked at those companies and they were not meritocracies. I’ll add “if you are inexperienced” to that quote and agree wholeheartedly.

How about being the lead author on all these papers? https://nicolefv.com/research

I haven't read any of them, nor done a google scholar search to see how many were cited, but it sure looks impressive to me.

> I’ll add “if you are inexperienced” to that quote and agree wholeheartedly.

100% agree that experience matters and can teach you when to ignore rules of thumb such as "prefer buying to building." But I've seen experienced developers succumb to the siren call of building their own, say, database pool, when they should not have.

> "make sure anything you build is a lot better than anything you can buy, because there are a lot of unexpected expenses in building".

The same thing works the other way as make sure anything you buy is a lot better than anything you can build, because there are a lot of unexpected expenses in buying".

Maybe I'm doing it wrong, but when the software I'm using, bought or built, goes wrong, my customers are mad at me, and look towards me to fix it. I might get sympathy if I claim it's a vendor problem, but they still want the problem fixed. The vendor may or may not be willing or able to fix the problem in a reasonable amount of time, regardless of if they're being paid or not.

My philosophy is not necessarily to avoid using outside software, but to be careful about what to use, because at the end of the day, I need to support the whole system, not just the parts I write.

That’s true but the lifetime costs of buying can be immense. On a home built solution you can maintain it for 30 years or more, on an off the shelf solution you may have had to rebuild and re customize 6+ times maybe even switch vendors

Maybe we are talking about two different things. 1) vendor-managed vs self-managed and 2) established implementations vs greenfield implementations.

This is the equivalent of a first year physics student saying Einstein comes off like a newb.

Nicole Forsgren is legit.

Fair enough! Maybe Twitter is just terrible for conveying meaning.

I think home-rolling is almost always the answer, unless:

- The IRL subject matter is distinct from your core business, programmatically distinct from your core in-house software, and generally the same across businesses (accounting, HR, etc.); or

- The product is open standard or otherwise highly interoperable, and has a permissive license (e.g., zlib, sqlite); or

- The product is highly complex and hard to get right (e.g., encryption, etc.).

In cases affecting your core product, with expensive licenses, and it's pretty easy to get it right, owning the software/platform outright clearly wins over incurring massive perpetual overhead just to make devs' lives possibly easier.

Reply on news.ycombinator.com