But isn't that in fact the fundamental problem we're talking about? You get security without knowledge exactly if you give up all control over the device. What's the point of "running" your own server if all control you have left is that you can physically smash it to pieces?
Yes you can take control, but only to the degree to which you acquire the knowledge to keep it secure. I think this is a very fundamental trade-off.