I don't keep anything incriminating or illegal on my phone (hell, I don't do anything illegal) but I'm terrified of the day where I'm in the airport, embarking on a £4k holiday (not insignificant in my salary bracket) and am asked to hand my phone and laptop/tablet over to confirm I'm not a terrorist. I'll be faced with acquiescing and compromising my principles or disappointing my girlfriend (who I almost always travel with) and having both of us taking a financial hit (and probably being added to a "list" or having my threat value increased).
It's not an unheard of situation in the UK, and is most often not intelligence-driven (more likely you are travelling through an airport where they are trialling some new counter-terrorist technology we've paid a small fortune to install).
I think a device with limited state (ROM-only?) which can give me reasonable assurances that no-one could tamper with it or install any government sponsored malware may be the solution. Although the utility of such a device is questionable, and it certainly wouldn't be fun to use.