Where does say Binary Blob end and Compiled to Silicon begin?

For example most CPU's are written in something like verilog or vhdl and include internal ram on the chip and include a prom with a little bit of code that handles start up tasks.

From what I can see Librem 5 is a general purpose computer with 2 sealed black box modem modules.

Yes, there can be any level of proprietary evil inside those seal boxes... and inside your disk drive controller of your PC.

The question then really is can that proprietary evil control the functions of your computer.

With the disk drive controller, probably pretty hard, with intel ME, probably pretty easy, with the Librem modems? My guess is probably pretty hard.