Some simple steps can be taken to protect data integrity and security:
-Don't leave the data connected to the Internet 24/7.
-Power off the storage medium when you are done with it.
-Keep it in a physically safe location when not in use.
-Encrypt the data or the volume.
-Have a backup of your backup on a different medium and if possible different location.
-Don't advertise the data.
These are activities which are easy to set up and can help insure a more pleasant digital lifestyle. You could go to more extremes but then it becomes invasive. Not worrying about it much because you find it unlikely or just don't care is bad. Not worrying about it much is what you want to do because you have taken the important but easy steps to keep yourself safe.
Having said that, I really am not worried about someone accessing most of these documents, and certainly not enough to go through the hassles of what you outline in your comment. If someone wants to go through the hassle of breaching my security so they can pour through my phone records for the last five years, or peer at my receipts for lunch at the restaurant around the corner, they're welcome to. What exactly are we worried about here?
Credit cards and bank account numbers are slightly more of an issue, but in the US your liability for theft from these sources is quite limited, even if you disclose the info. Plus, my bank account number is on the bottom of every check I write. What kind of scheme is that, anyway?
Don't get me wrong, I shred any paperwork with personal info, and I use encrypted services for backups, etc. But I don't lose any sleep over the worst case scenario of someone gaining access to this data.