Hacker News Reader

Comment by jb_s | Hacker News Reader

C# was never really slow in the same way as python, etc. And anyway 99% of the time you're gonna be slow because the sack of meat writing the program screwed up some aspect of the system design or the code. Unless you're using something really shit-tier for perf.

The gap closed significantly with .NET core, which is why everyone was quite surprised when .NET 5 (the next iteration) had a fairly significant speedup in many scenarios.

For reference, Stack Overflow was running on .NET MVC on like 2 servers pretty recently (with some auxiliary infrastructure for CDN and search) and using MS SQL.. I think it might still be running on this setup but not 100%. Honestly I have no idea how they do it on a .NET monolith but there you go.

View on HN

Stack Overflow runs on 9 web servers with (iirc) 48 logical cores (2 x 12-core Xeons) and 64GB RAM. Those servers are shared by a few apps (Talent/Job, Ads, Chat, Stack Exchange/Overflow itself) but the main app uses, on average, ~5% CPU. Those machines handle roughly 5000 requests/sec and were running .NET 5 as of September 2021 (when I moved on). That’s backed by 2 v. large SQL clusters (each consisting of a primary read/write, a secondary read-only in the primary DC and a secondary in the failover DC). Most traffic to a question page directly hits SQL - cache hit ratio tends to be low so caching in Redis for those hits tends to be not useful. As somebody mentioned below, being just a single network hop away yields really low latency (~0.016ms in this case) - that certainly helps being able to scale on little hardware - typically only 10 - 20 concurrent requests would be running in any instance at any one time because the overall request end-to-end would take < 10ms to run.

Back in full framework days we had to do a fair bit of optimisation to get great performance out of .NET, but as of .NET Core 3.1 the framework _just gets out the way_ - most memory dumps and profiling subsequent to that clearly pinpoint problem areas in your own app rather than being muddied by framework shennanigans.

Source: I used to work on the Platform Engineering team at Stack Overflow :)

That's some great info, thank you!

> cache hit ratio tends to be low

That's surprising to read. Is that because of the sheer volume of question pages? I don't think I've ever been on an SO page that couldn't have been served straight from cache.

Is it? Most people come to SO from Googling their random tech problems/questions. Not sure how much value there is in caching my random Rails questions, etc

I would expect SO usage to follow a distribution like Zipfs — most visits hit a small subset of common Q/A, and there’s a ridiculously long tail of random questions getting a few visits where caching would do next to nothing. I’m fairly positive I’ve seen some post showing this was true for atleast answer-point distributions.

Though I guess it’s possible for a power distribution for page-likely-to-be-hit to still be useless for caching, because I think you could still get that distribution if 99% of hits are on nearly-unique pages; with a long enough tail, you’d still have only relatively few pages worth bothering to cache, but by far most visits are in the tail

> Stack Overflow was running on .NET MVC on like 2 servers pretty recently (with some auxiliary infrastructure for CDN and search) and using MS SQL.

They are absolute beasts of machines. But yes, it’s incredible what you can do with just a few colocated physical servers.

It suggests to me lots of dev shops pay the IaaS or PaaS cloud tax, long before they would have hit any scaleabilty walls with a non-cloud setup.

A poster above claimed the servers were “48 logical cores (2 x 12-core Xeons) and 64GB RAM”, which really isn’t what I would consider such a “beast” of a machine when the RAM is in laptop territory, and a modest number of cores for a server.

Nowdays you can purchase that machine on the second hand market for something like $200-$300.

They are measurably faster than even contemporary laptops though plus you often get ECC ram and raid disk setups and the good old Xeon didn't used to ramp up and down in speed, it just ran fast all the time. I'd still characterise that as a beast, especially on $/performance terms (although the power consumption is a worry).

Xeon covers a pretty wide variety of chips. Of course, the Pentium II Xeons didn't have speedstep either. 12-core tells us either fairly high end but older or kind of medium-low but newer. Dual socket tells us not the really low end Xeon that shares a socket (and probably a lot more) with the high end enthusiast desktop chips.

Don’t we have server machines with multi-TBs of RAM nowadays?

Late reply, but you make a good point. Perhaps I am thinking five years ago, and back then the specs seemed more impressive.

Also, I hadn't twigged that “logical” cores are distinct from physical cores, which made the core count seem more impressive.

> And anyway 99% of the time you're gonna be slow because the sack of meat writing the program screwed up some aspect of the system design or the code. Unless

Or in my experience, most of the time a service's latency is dominated by out-of-process calls, e.g. the time taken to talk to other services over http, or to retrieve data from a data store. Speeding up the runtime is welcome, but even a massive 40% speedup of something that constitutes 10% of your total latency is ... closer to a 4% reduction in latency. Design matters more.

If your program is slow because it's waiting for external service response, you're doing programming wrong. Your program should do other work in the meantime. I guess you're already doing that, but if so, doesn't that invalidate your reply here?

> you're doing programming wrong. Your program should do other work in the meantime.

It's not always true that there is other work to do in the meantime. In fact in my experience, it seldom is. "you're doing programming wrong" is a very strong statement, and not one that I take seriously in this context.

Typically you "await" the external service response, so that it is not using a thread to do that, and "other work" in the form of starting to deal with other requests can happen in the meantime, thereby increasing service throughput.

But that won't speed up a given request - you can wait for an external service more efficiently, but you can't wait faster.

Services that do not depend on any other http services or any data store do happen, but they are rare in my experience (calculation engines, I suppose). So for almost every service, when thinking about response time, you have to, first and foremost, think about the latency of the data stores or upstream services.

That is all true, but in the context of .NET performance, it's not relevant.

What then is, in your opinion, most relevant?

Time taken by processing done by the application itself. Faster processing is good even if your backend is mostly waiting for other services - now it can wait for more services at the same time and deal with their output faster, allowing your app to use less resources or handle more traffic. That is especially pronounced on scale - if you're spending X million for computing time a month, reducing it by few percent is very interesting to you.

Both can be relevant. In the real world, designing how to deal with dependencies and data stores is clearly relevant, as it can be the largest large part of the time taken to respond to a given request. It would be a design error to ignore it.

Yes, but external services are irrelevant to performance of . NET.

If your job is to make performant services, in .NET (or in any similar language) external services are 100% relevant to your job.

If You want to be pedantic - and you most certainly do - then only .NET performance itself is relevant to performance of . NET itself, that's a truism, as defined.

But this narrow focus is not useful - if you want to do the job, then you have to think a bit more widely, and understand what the real problem is.

Sorry but this is not pedantic at all. You're saying performance speedup is irrelevant because most time is spent waiting for external services... But that is simply not true if you're beyond just a few servers. Output of external services needs to be processed and having faster software means less resource is spent per request, thus more requests can be served for the same resources. That is very important.

> You're saying performance speedup is irrelevant

I am not saying that; you are oversimplifying into a straw man. great-grand-parent comment is where I literally put a non-zero number to how relevant language perf improvement was: https://news.ycombinator.com/item?id=29295950 and later on "Both can be relevant"; which all contradict your characterisation.

The only person who said above "it's not relevant" is you, and you also said "external services are irrelevant" - You're projecting the "it's irrelevant" statement onto me here.

But the design considerations of how and when to use external services very definitely are relevant to service latency, contrary to what you say, for reasons given multiple times above. Your current odd comments are not fact-based or interesting, so I don't think that you have anything more to add to this discussion at all.

>If your program is slow because it's waiting for external service response, you're doing programming wrong. Your program should do other work in the meantime.

What work? Mine bitcoins while you wait the result for an API call?

Usually processing other requests

I think any sane person here assumes any IO, especially in .NET is already async and allows other requests to use that time… Most web apps are still IO constrained.

Sure, but how is waiting for external requests relevant in any way to .NET performance then?

If you're asking "how is waiting for external requests relevant to my service's response time" the answer is "because it's usually the largest part". I really don't know how to explain it more simply than that.

If you need to improve this, and that code is .NET, then the solution is a different design, also in .NET Code.

I am asking how is waiting for external services relevant to .NET processing time.

I refer you to my answer above. https://news.ycombinator.com/item?id=29306220

> Honestly I have no idea how they do it on a .NET monolith but there you go.

Low latency in a single rack can work wonders for performance. All those cloud services talk to each other over miles of cables and if you can slash latency to submillisecond regions, you get less wait time and free resources quicker. If you don't distribute your state across multiple Microservices, you can also save quite a bit of overhead.

Plus hardware is just wicked fast nowadays and SO has a model of millions of reads for a single write.

Looks like SO's architecture is still pretty simple, yeah. https://stackexchange.com/performance

Reply on news.ycombinator.com