Hacker News Reader

Hacker News

Comment by jyounker | Hacker News Reader

Full thread

jyounker

Those seem like extreme positions. Reality is more like this:

1) Real engineers write systems that accomplish the organization's goals.

2) Most people don't need to write large distributed system, but they will end up writing small distributed systems.

3) Small distributed systems can be surprisingly complicated.

View on HN

2) is because making a distributed system is more interesting and fun than debugging why your ORM making inefficient DB queries.

>why your ORM making inefficient DB queries

It's also crazy that we still have this and other basic problems with databases. Frontend and backend is moving so fast, with so many fun technologies, but even backing up mid-size database, testing backup and restring it, isn't a trivial problem...

The market demand for database work is primarily not in making mid-size databases work better, but in making the "mid-size database" abstraction work on increasingly large (and increasingly distributed) datasets. I'm familiar with large companies who routinely run SQL queries over ~all their logs; a decade ago that would have been somewhere between extremely challenging and impossible.

When you have three folders stuffed with hundreds of files that are laced with your ORM annotations the ship has already sailed.

This is so true. Hire a bunch of senior engineers to solve junior level problems and they'll turn the junior level problems into senior level problems :p

I'm mostly surprised how developers as a community can both:

- Espouse we really need to care about communication in every form

- Still manage to let a few 'seniors' capable of solving the problem do so, at the cost of destroying most momentum by making the solution incredibly difficult to reason about and leaving next to zero documentation behind.

Then again, when business doesn't give the time to fix this or doublecheck, it's no wonder. And then hiring managers look cross-eyed at developers for wanting to work on green project instead of the next dumpster fire

I'd say that you often want a small distributed system for availability reasons, even if scalability is not a concern.

If you've got an ORM, you almost certainly have a distributed system already (arguably not if your ORM is talking to SQLite in process). Even if DB and application are on the same host, I'd call it at least a potentially distributed system.

I'll do the de-minimus anti-ORM screed here and just say if it's harder to get an ORM to make efficient queries than it is to write efficient queries by hand, maybe an ORM isn't a necessary abstraction layer.

That only happens the first time though. Distributed systems are the devil when it comes to debugability.

Some examples of small interesting distributed systems that occur even with a traditional stack:

With multiple visitors on a web app on top of a relational database, you have a small distributed system if they can somehow interact. Even something as simple as a web shop can leave you with an interesting question like what to do when multiple people ordered the last item in stock.

Even just a page with a little bit of ajax going on can give you an interesting small distributed system where a single visitor can interact with themselves.

Or say you need to sync up information between two systems, e.g. export information from the ERP system to a web app or vice versa. That's a small distributed system - there are lots of pitfalls in state transfers.

How is Position 2 extreme? It's just a fact that the vast majority of sites don't need highly available, massively distributed and scalable systems. The small amount of organizations that actually need something like that are the rare exceptions.

This is...a refreshing take. Good shout.

Reply on news.ycombinator.com