> For certain things in signal you need to trust the server hardware and suspicious technologies like SGX. SGX is a prime target for a NSA backdoor. That some upper-layer software there is open source (with lots of caveats, e.g. you cannot verify which server version is really running and signal has a history of lagging behind on publishing their "open source" version of the server code) is a distraction at best.

Before you worry about super expensive hard to remove hardware bugs, you should worry far more about the software running on that hardware. It is much more likely that theres bugs in the software than the hardware.