Hacker News Reader

> Anyone who says to me "I'll just put everything in PostgreSQL and optimize later" sounds just as naive as the person who says "let's build for scalability before we have any customers". Both are giving themselves problems they don't need

Both are giving themselves problems, but they are fundamentally different decisions. I will almost always take the "optimize later" approach as there's not always a "later".

For most new products, the biggest challenge is finding product market fit. Once you have that, the world is your oyster. I will happily pay a 10% tax after PMF to be 10% better at finding PMF now.

If I've found PMF, I will have more revenue, more traction, more budget, etc to fix the problems I created. If I don't have PMF, everything goes in the trash.

It's not 10% though, is it.

10% was just a smallish number. In my experience, it's actually less than 1%.

Code typically follows one of two paths:

* It's in an unused/underused part of the app. Performance doesn't really matter.

* It's in a highly used part of the app. Performance matters, but you'll never be able to predict the features that get layered on top of it. Fix performance as you build those new features.

This is my experience. Get it out there, learn what you don’t yet know about the problem/product space, then improve and optimize.

The hard parts, in my experience, are being sufficiently seasoned to know what parts are hard to do later and are worth facing up front, and making sure the decision makers buy into this so you’re not blindsided by the classic “there’s no time for tech debt.”

A new business already has a laundry list of "I need to ${thing} to survive". Being able to move one such problem a couple of years in the future is already an attractive value before considering most businesses won't last that long anyways (for reasons unrelated to how optimized the database was). I mean if you already have the know how, confidence, and experience to do it right in the same amount of time then duh. If not, is spending precious extra time that early on really going to help you survive long enough to reach where how optimized you infrastructure spend is seems like an important problem for your cash flow?

Sometimes I think this then gets conflated with more established businesses where continuing to start every project so heavily this way does tend to create more problems than it solves.

I once ran the daily trade reconciliation of an entire US primary exchange on my laptop, using a PERL script (in 2007). I wrote it in a couple of days, and each day's processing ran in minutes. Ten years later, I witnessed a team spending weeks spinning up spark clusters to perform a job that was significantly smaller. At the time I thought I was the person who was missing some important context. Now I'm beginning to realise that I was not in fact misinformed or insane.

Maybe you were missing something? You could have been a manager, commanding a team of Spark engineers and an even higher salary?

... then they see the cloud bill.

Cloud companies found a way to directly monetize developers' love of complexity and overengineering. It's genius.

Developers often love simplicity and underengineering, which can also lead to performance problems.

My personal definition of engineering is: choosing good compromises between competing needs.

This comment will be taught in business schools one day.

1. Find pathology.

2. Attach toll booth.

3. Profit.

> Cloud companies found a way to directly monetize developers' love of complexity and overengineering. It's genius.

The developers' tastes differ a lot. Consider the extreme counterexamples

* the "small code" scene

* people who love to port modern games to C64, Amiga, retro consoles, ...

The situation is in my opinion rather that quite some managers love complexity and "future-proofness", and thus hire developers who have a similar taste.

IME it's idiot managers that go for love of complexity and overengineering - though not all mgrs are idiots.

Especially in small/medium sized businesses, hardware is cheaper than payroll. It's most cost effective to pay people to build features (that drive revenue) while paying a bigger hardware bill than spend engineering hours optimizing hardware usage.

At some scale they'll be a break even/shift

> "I'll just put everything in PostgreSQL and optimize later"

Sounds good to me!

"I'll just drink and eat heavily in my 20s and worry about exercising later"

Or you could... not?

To what benefit? In my (early) twenties, I could do that without consequence. But what benefit is there to not doing it while you can? (I mean, assuming you're doing so at a non dangerous level)

I'd ask what's the benefit in drinking, but most people see brain destruction and increased risk of doing something stupid as "fun".

Alcohol also reduces the level of social awkwardness quite a bit that many people in the software engineering field exhibit. Which can be quite helpful in making new friends or finding a partner. I'd expect a significant percentage of people would not have been with their current partner without alcohol.

> In my (early) twenties, I could do that without consequence

I mean, maybe for you, anecdotally. Anecdotally, I had a friend at 23 who almost died due to his unhealthy drinking and his body basically shutting down. Doctors told him he couldn't drink anymore. And studies have shown that any amount of binge drinking/eating is bad for health outcomes long-term.

You're actually proving my point... people just hand-waive away warnings about future problems, using cherry-picked "evidence" like "hey my friend binge-drank for 30 years and he's fine", or "my father smoked two packs a day for 50 years and he's still around".

Similarly, (ab)use of technology for the wrong thing sets you up for problems later on. Don't do that and you avoid the problems later on.

But nobody wants to listen to the doctor telling them to drink less or give up smoking. Just like nobody wants to listen to good advice from experienced technologists. Sigh.

You know people drink and exercise at the same time right? :)

Agreed. Way easier to optimize this one thing than redesigning your application every few years...

> They think, it's fine to be inefficient, I'll just get a bigger X. And we are now blessed with technology and solutions that make that a possibility, when for the longest time it wasn't.

Unfortunately, the culture in my group is basically this. We never address things until they're dire inefficiencies; thus, I spend a nontrivial amount of my time addressing workloads that take 10+ hours but need to be faster (and often with minor changes are cut down to 2 hours).

Having a culture of "I'll just throw more hardware at it" / "If it takes an hour or ten, I still am going to run it and get the results tomorrow" is costing ridiculous amounts of money and lost time. It's such an uphill battle to try to change that culture.

In my fuzzy sentimental memory, software bloat seemed to always self-regulate to 5-10 lines per second of logs or 2-3 screen flickers per minute, whichever applicable. Things go slower and optimizations come about, go faster and another Midleware.js comes to be.

My working theory is this is the common denominator max comfortable flickering rate for developers. There's got to be sweet spot for UI, how do I say, fluidity. And that fluid-ness usually aren't controlled for dev tools, tools themselves must evolve to stay at comfortable refresh-rate, by natural selection. That shouldn't be too weird of an idea.

I don’t know if you’ve ever had to build an MVP at an early stage company