Hacker News Reader

Comment by tgtweak | Hacker News Reader

Cool but how much does one of those servers cost... 120k?

Pretty crazy stuff though, would love to see something similar from cloudflare eng. since those workloads are extremely broad vs serving static bits from disk.

View on HN

I think the described machines were designed for a very specific workloads even in CDNs world. As I guess they serve exclusivelly fragments of on-demand videos, no dynamic content. So high bytes to number of requests ratio, nearly 100% hit ratio, very long cache TTL (probably without revalidation, almost no purge requests), small / easy to interpret requests, very effective HTTP keep alive, relatively small number of new TLS sessions (TLS handshake is not accelerated in hardware), low number of objects due to their size (much fewer stat()/open() calls that would block nginx process). Not talking about other CDN functionality like workers, WAF, page rules, rewrites, custom cache keys, no or very little logging of requests etc. That really simplifies things a lot compared to Cloudflare or Akamai.

https://www.thinkmate.com/system/a+-server-1114s-wtrt

For a EPYC 7502P 32-core / 256GB RAM / 2x Mellanox Connect-X 6 dual-nic, I'm seeing $10,000.

Then comes the 18x SSD drives, lol, plus the cards that can actually hold all of that. So the bulk of the price is SSD+associated hardware (HBA??). The CPU/RAM/Interface is actually really cheap, based on just some price-shopping I did.

The WD SN720 is an NVMe SSD so there are no HBAs involved it just plugs into the (many) PCIe lanes the Epyc CPU provides. CDW gives an instant anonymous buyer web price of ~$11,000.00 for all 18.

Surprisingly less it seems? Those NICs are only like a kilobuck each, the drives are like 0.5k, CPU is like 3k-5k. So maybe 15k-20k all in, with the flash comprising about half that?

Seems surprisingly cheap, but I’m not sure if that’s just great cost engineering on Netflix part or a poor prior on my part … I’ll chose to blame Nvidia’s pricing in other domains for biasing me up

That’s in the right ballpark, I believe.

Reply on news.ycombinator.com