Comment by tschellenbach | Hacker News Reader

Comment by tschellenbach | Hacker News Reader

Parent Full thread

Most common mistakes I see people make:

- They ignore overhead. If an engineer makes $100k they calculate with those number. The reality is a 2X to 10X overhead depending on the company. - Required return. You can't spend $100 to make $100, no investor will fund that. So you need to do activities that generate an adequate return. - Opportunity costs. So say that you have an engineer that costs 100k, the overhead takes that to $400k and they need to generate at least $600k a year. That still doesn't mean that you should do the project if you have something else that's even better. - Maintenance. People always underestimate the cost of maintenance. I'm skeptical of any estimate where maintenance is < build costs

One thing I regret building is an analytics pipeline. We should have relied on Segment for that. I've also once built an analytics platform from scratch which was bad.

On the flip side, one time we bought an ETL tool and it was terrible compared to in-house solutions

The common mistake I see people make is presuming that the "buy" will deliver on their end of the deal. When they don't, that engineer making $100k is spending a lot of time (i.e., and money) writing support tickets, desperately trying to get it escalated to someone on the vendor's side who can actually do something, as inevitably you have to first peel away the layers and layer of customer support, account managers, "solution" people who aren't actually capable of fixing anything. (Even for some inanely simple questions, sometimes, things that I think "wow, a support rep. should actually be able to answer this, it's not a highly technical problem for once" and then, nope, it has to get escalated to eng., inevitably.)

The amount of my own salary that has gone towards "Azure support ticket monkey" is frankly frightening. And I have never once seen that included in the estimate for "buy".

Many of the "build" solutions I have run require <1 engineer to maintain. When they require maintenance (whether due to something needing an upgrade, new feature request, etc.), yeah, there's maybe a two or three week piece of work, but then there are months long spans where it just sort of hums along in the background.

Even outside of "support", I still end up having to dedicate eng time to "buy" solutions, to fill in gaps in their implementation. (E.g., an artifact store having read-your-writes bugs. Heck, Github Actions ("buy" of CI) has so many bugs in it that we hit in the course of adopting it…)

You are correct in the fact that 'buy' doesn't mean zero cost. At scale both 'build' and 'buy' suffer from the same problem, overhead.

'Build' has the overhead of the initial development, documentation, testing, the follow-on maintenance, refactoring, etc. If you have one 'build' item in your code base that's not a huge amount of overhead in the long term. If your entire code base is custom you have a scaling problem; you will end up spending a lot of your development effort on this overhead rather than revenue generating work.

Ideally 'buy' lets you go further but you will eventually reach the same scaling problem. The overhead here is in research, prototyping, integration, bugs, workarounds, etc. As you have more and more external dependencies this overhead goes up until you are in the same boat as above. Also not all dependencies are created equally; some have more overhead than others.

For background I do embedded software where basically everything is 'build'. When I say everything I mean file systems, network stacks, hardware abstraction layers, communication protocols, testing frameworks, and so on. I would estimate that roughly 90-95% of what my team does is maintain all of the 'build' pieces, none of which makes a profit but all of which are required for the features that do make a profit; we just don't have time to work on the latter unless we take more shortcuts in the former. So when you say that a 'build' solution takes < 1 engineer to maintain consider what that looks like in 5, 10, and 25 years. The overhead stacks up, the industry changes, institutional knowledge is lost, hiring and onboarding becomes more difficult, and so on. Of course my bias is in systems that last a long time where it's incredibly difficult to replace custom bits that have outlived their usefulness with third party bits.

what was the ETL tool? Fivetran is great at EL, but leaves so much work on T. They have some open source dbt models[0], but they need a lot of work. For example, the Hubspot model doesn't have a way to join most of the tables (e.g. you can't join Deals and Companies, which seems like an obvious thing you'd want).

If they dedicated one data engineer to building an amazing set of dbt models on top of their raw EL output they could drastically improve the lives of their customers.

0 - https://hub.getdbt.com/#:~:text=feature_store-,fivetran,-ad_...

This is a great explanation of overhead and opportunity cost. I'm always trying to make that pitch for selling my product, but the biggest issue is that I think many companies are just jumping in on BUILD without even researching BUY.

That's wild, at every company I've worked at so far the default was always to BUY. That meant dev teams needed to fight for BUILD and fight for their existence often.

I'm thinking it depends on the purpose of the software and how much customization and integration is required. A peripheral application may be a stronger BUY than something that is deployed at the core of the business process.

Reply on news.ycombinator.com