I have 6 and ended up needing to use my gaming PC for a build server.
I think you could get by with 1 computer, but it’ll have to have a pretty decent machine.
Between agents running tests, CI, docker image builds, an average $400 mini PC won’t cut it.
Don’t forget also many older mini PCs don’t support KVM. Some newer ones don’t support AVX/ mongodb.
It’s not so easy to buy any old hardware sadly.