Bare bones hardened kernel + shim init + target application are all you need, and will be your highest security/reliability systems.
And yeah, threadrippers are a must.
In security focused orgs though you review all code yourself with the exception of things with extensive third party signed review such as the Linux kernel itself. Even then I review codepaths in the kernel critical for my use case such as random.c
From there, if I -alone- compile containers, kernels, or binaries, someone could coerce me to tamper with them to compromise all downstream users. Same if there was a central build system I can access. To mitigate this I ensure my artifact builds are deterministic, sign my changes, and have team members review my changes, reproduce my artifacts bit for bit, then counter-sign the results.
It is never wise to be in a position where there is possibility of you yourself tampering with things that control anything of value, or else someone will coerce you to help them steal said value.
As a security engineer it is my job to ensure no one ever has to trust anyone, including me.
Interesting thread!
Sure, compromised binaries are nasty but personally I do place quite a lot of trust with the distribution repos.
(PS, if you are reading this and contribute packages to distribution repos: Thank you!)
Security comes down to reducing attack surface, ideally to an infinitesimal degree.
I mean, they give a wonderfully higher quality of life for compiling but they are $2,500-6,000 just for the CPU. Of course this can be done on much lower-cost CPUs! A 12-core Ryzen is already at the "I have lots of money to spend" end of what I'd suggest for something like this.