It's an interesting point that you make, but I think the analogy is generally flawed for most people.
Most people don't carry around a significant amount of personal information of others (I have an electronic address book, but I don't have my friend's credit card/banking/social security numbers).
This rings rather personally to me, and I do consider it an absolutely jerk move. Of course, being irresponsible with other peoples' information is also a jerk move. In 1999, an HR rep copied into Excel the entire contents of the employee database (something he knew was not acceptable practice). A disgruntled support tech discovered this while rummaging through his computer and posted all of that data on a public, searchable web site. At the time (and this may still be the case) it was not illegal to post the information that he did, which included social security numbers, mother's maiden names, birthdays and a bunch of other details. It took a few years to get the site properly shut down, to catch the guy, and to prosecute him for the theft of the data. It's been a fun decade closely monitoring and correcting my credit reports.