Hacker News Reader

Comment by boznz | Hacker News Reader

> FreeBSD supports up to 1024 cores on the amd64 and arm64 platforms.

Sounds pretty future proofed unless I'm missing a x86 processor out there that does this

If you combine multiple CPU sockets, you could get there. EPYC, IIRC, supports up to 64 cores per chip, so if you build a machine with 16 sockets, you get 1024 cores. To my knowledge, no such machine exists today, but HPE offers (or used to, anyway) a machine with 32 Xeon chips, so its core count could well reach several hundred. (I may or may not be drooling at the thought.)

    Intel® Xeon® Platinum 8490H Processor 
    Total Cores 60
    Total Threads 120
    Max Turbo Frequency 3.50 GHz
    Processor Base Frequency 1.90 GHz
    Scalability S8S

Up to 60 x 8 = 480 cores or 960 threads

    AMD EPYC™ 9754
    # of CPU Cores 128
    # of Threads 256
    Max. Boost Clock Up to 3.1GHz
    All Core Boost Speed  3.1GHz
    Base Clock 2.25GHz
    Socket Count
    1P / 2P

Up to 128 x 2 = 256 cores or 512 threads

It's a lot easier to build such a machine if you have something to run on it, I suppose.

> up to 1024

Curious where this (rather large, yet still seemingly arbitrary) limit comes from.

> Curious where this (rather large, yet still seemingly arbitrary) limit comes from.

It is Good Enough for now, while keeping various pre-allocated, statically created structures with-in reasonable size limits:

> Global and allocated arrays sized by MAXCPU result in excessive bloat on systems with lower core counts. In addition, some code used u_char (8 bits) to hold a CPU index, which is not valid if MAXCPU is greater than 256.

> A number of recent commits addressed these sorts of issues, including at least: […]

* https://cgit.freebsd.org/src/commit/?id=9051987e40c5

See:

> The SMP system now supports up to 1024 cores on amd64 and arm64. Many kernel CPU sets are now dynamically allocated to avoid consuming excessive memory. The kernel cpuset ABI has been updated to support the higher limit. 76887e84be97[1] d1639e43c589[2] 9051987e40c5[3] e0c6e8910898[4] (Sponsored by The FreeBSD Foundation)

* https://www.freebsd.org/releases/14.0R/relnotes/#kernel-gene...

Gotta have some limit, 4x the current limit of 256 seems reasonableish. Dual socket Epyc 9654 is 96 cores * 2 threads / core * 2 sockets = 384 threads. Intel says their Xeon Platinum 8490H can live on an 8 socket board[1], if you can find one (SuperMicro has one, no price listed [2]; not sure if this is really an 8 socket system, or if it's 4x dual-socket nodes in one chassis?); 60 * 2 * 8 = 960, so that's within the limit, and 8 socket boards are pretty difficult to find.

[1] https://ark.intel.com/content/www/us/en/ark/products/231747/...

[2] https://www.supermicro.com/en/products/system/mp/6u/sys-681e...

9754 is 128/256 now, so 256/512 for that.

That supermicro system is 8-way; it's 4 dual-socket motherboards but they're one system, hooked together by backplane boards. You can price supermicro's complete-system-only stuff (all of it now, alas) out on thinkmate or similar sites, but a minimal config (and you'd never buy that for a minimal config) hits around $60k.

Bitmaps for logical CPU cores and certain lock-free algorithms don't scale well to arbitrary high CPU counts e.g. reclaiming resources once in a while is O(n^2) or worse or the size of the lock structure is linear to the maximum number of cores etc.

The relevant parts of the ABI have been future proofed to allow raising the kernel CPU core count limit without breaking the syscall interface for systems with less cores than the existing limit.

"supports up to" doesn't have to mean "works well/optimally with".

Reply on news.ycombinator.com