Even with a rooted device where perhaps you personally coded up the ROM you are still missing a piece which is the binary blob that runs the baseband radio. That firmware is, afaik, not something which exists in any sort of open-source or rootable manner. It's a closed blob running proprietary software on your phone, and it runs at a lower level than the ROM/OS does. So, even if you go to great lengths to secure most of the software that runs on the device (a noble goal, it's your hardware after all!) then you still must contend with the uncertainty and perhaps risk (depending on your threat model) of that untrusted code running there. You can search around the web for articles covering baseband radio exploits that span the years...