System Design Primer: Learn how to design large-scale systems | Hacker News Reader

System Design Primer: Learn how to design large-scale systems

github.com

170 pointsjaboutboul29 comments

I thought this was going to be about system engineering, but it's just about web scale distributed computing. Disappointing, because I'm digging into systems engineering at the moment and got a bunch of books on it for Christmas, plus thumbing through the SEBoK wiki [1] and associated content.

1: https://www.sebokwiki.org/wiki/Guide_to_the_Systems_Engineer...

Any books you’d recommend on the subject?

Not yet, though I'd love to do recommend some if I could - as I say, I've only just started looking into it fairly recently!

If you're interested, drop me an email (in my bio) and I'll let you know, once I've read my new books, if they're any good.

Could you list the books you bought? Thanks!

Sure thing (though it was my lovely wife that bought them, not me!):

- Systems Engineering Demystified by Jon Holt

- Essential Architecture and Principles of Systems Engineering by C. E. Dickerson & Siyuan Ji

- System Design & Management: An Introduction to Systems Engineering by Bob Parkinson

On a somewhat related note I also got Difference & Repetition by Deleuze and Process and Reality by Alfred North Whitehead, but they're more philosophical than practical.

Awesome, thank you!

Funny how the term "systems engineer" is loaded. I though more along the lines of https://diveintosystems.org/

Honestly I think it's a bit silly that such a broad term as "system" is often used to describe software and hardware administration. Reality is made of layered and networked systems so it feels odd to me to call somebody who manages Linux servers a "system administrator".

Might as well call it "thing manager".

lol so true. Anything that does something is a system, be it hardware, software, biological, mechanic...

It seems like all such resources are about web-scale distributed systems and the content/methods are quite standardized.

What about other large-scale systems? E.g. desktop applications - for instance, how to design a large-scale desktop system such as Adobe Photoshop or MS Office.

Systems engineering is the discipline that got us to the moon, and builds advanced technology like military hardware.

For other large scale software projects, I think they tend to use enterprise software architecture patterns like service-oriented architecture (basically microservices) and ESB (basically kafka).

I doubt you could find two that look the same on the inside, too many talented coders are needed to get to that point; but there's some good stuff by Sean Parent of Adobe on YT.

this resource exists purely for people to practice for systems design interviews. or at least, that's 99% of the use cases of this repo

Architecture of open source applications, which has a free book version available on their site, has that info on some other large desktop applications .

More like 'learn how to convince someone that you know how to design large-scale systems'

Past related threads:

Learn how to design large-scale systems - https://news.ycombinator.com/item?id=17522362 - July 2018 (185 comments)

The System Design Primer - https://news.ycombinator.com/item?id=13823979 - March 2017 (57 comments)

Thank you

HN already provides a link to past threads. Check out the "past" link right next to the submission.

Thanks, but "past" link does not show past discussions of the posted article, just older states of HN.

> Thanks, but "past" link does not show past discussions of the posted article, just older states of HN.

No, not really. It shows a list of past submissions, which of course include the corresponding discussions.

Take a look: https://hn.algolia.com/?query=System%20Design%20Primer%3A%20...

There are two completely different “past” links on the page, hence the confusion. One shows previous days, other previous posts.

I never realized there was an article scoped 'past' link.

That's the search box at the bottom of the page. I get that "hn.algolia.com" link when I search for the title of the article. Otherwise I don't see a "past" link. Maybe there is, I don't know.

You understand that you just lectured the moderator of HN on how to use the website?

HN already provides a link to profile bios. Check out the %username% link on top of the comment. /s

TBH Dang’s bio has only implicit connection to The fact that he is a moderator here, but most people of course know.

Yup and I'm glad you know that because I wish it got more attention! (I'm a mod here, sorry that wasn't clear.) The reason I post those links to past related threads about specific stories is that many readers like to look up old comments on the same topics. Often there are surprises and gems in there.

Oh and there's no implication that the story shouldn't be reposted—reposts are fine after a year or so (https://news.ycombinator.com/newsfaq.html).

Btw, the 'past' link in the top bar shows you (more or less) what the front page looked like on a particular day. To find threads about specific past stories, it's best to use HN Search at the bottom of the page.

I think this is very valuable and helpful.

What really matters I think is when you really need to go this route? Wat what kind of traffic / transactions/s level do you need to (re)design for this?

Or even, by what metric would you opt for this solution?

I mean, just buying a bigger instance will get you a long way and if you offset that cost to development time / effort / lost opportunity, how will the scale be tipped?

I think it depends a lot on the application use cases and requirements.

Things like:

Will there be spikes in demand 50x higher than baseline usage? I.e. how fast and efficiently the system needs to scale up?

What level of reliability is needed? I.e. is it really a big problem if the system goes down for an hour a handful of times throughout a given time period?

Nah people need to go this route just to pass the interview and get the $600k job by answering the question made up for the interview - it always need to scale globally with at least 99.99 availability and development time /other non-tech dimensions almost always not in scope

Reply on news.ycombinator.com