But one of the biggest challenges is going to be privacy...
I'm a runner, so diet is important. For 9 months I tracked everything I ate. Weighed portions, counted calories, logged everything. The data was insightful. I learned that I need to eat 3,300 calories a day to maintain my weight, with a range of +/- 300 calories.
After 9 months I stopped tracking things because it became time-consuming. Partly was because I learned what it "felt" like to be full, but mostly because tracking it all was really annoying. If all of that could be automated somehow, I'd love to have a spreadsheet of data which I could graph and chart. "At 8:16am, you drank 7.8oz of Orange Juice, totalling 108 calories." Cool, my O.J. container and glass measured that automatically, and uploaded it to my database!
But right now doing it is extremely manual. All of Wolfram's amazing data comes from some sort of automated system he setup once and never worried about. A pedometer. A script chewing through 20 years of archived emails. That form of automation doesn't exist for many types of data people would like to collect, such as food intake, or blood sugar testing for diabetics.
Inventing these non-invasive automated sensors is key for society-wide personal data collecting. Otherwise most data gathering will be sentenced to the dedicated few, rather than the busy many.
I agree with you that measuring is the most intensive task of tracking personal statistics. I can enter how many servings I had of what in any fitness tracker website, but what if I had 1.3 servings? What if I had .2 servings (just a taste)? How do you estimate that without measuring it first? And if you're measuring everything on a scale, where has the fun gone in life? What's the difference between medium intensity running vs high or low? If I biked at 15mph it might be medium, but what if I kept that 15mph steady up steep hills? Or what if I kept 15mph down a hill without pedaling? We're built for rough estimates and are amazingly adept at convincing ourselves we're right.
I have enough difficulty with time tracking at work either breaking my productivity or cutting into my limited "me" time, and that's just a petty little thing. I usually end up estimating at the end of the week then fixing discrepancies that always come up.
When I was looking into it, I noticed that a lot of the blood sugar testers were electronic and could record the series of readings for you. You still had to prick yourself, but it looked pretty easy.
(Sadly, the one non-invasive blood sugar recording device I found was apparently discontinued for causing electrical burns.)
It just occurred to me that the new breed of payment services (Square, Google Wallet etc) should combine with food vendors to offer this service.
In Australia at least fast food restaurants publish the calorie count of each food option. It wouldn't be impossible for the payment company to do the correlation.
Also sports (and weigh loss) food vendors should embed NFC chips in their packaging so you can scan them before you consume them. There are plenty of cross-promotion possibilities there...
"Google Takeout" indeed.
> And as I think about it all, I suppose my greatest regret is that I did not start collecting more data earlier.
Hopefully he can change that. For example, his observation about the 7% use of the backspace key is an obvious one. If he eliminated backspace entirely, he could speed up typing by as much as that much - nontrivial given how much time he apparently spends on email! An experiment would be using some typing tutor program for a few months and seeing whether the keylogging notes a decrease in backspace.
Why should you feel regret? There is nothing more human than communicating with other people!
Why are predictive keyboards so popular on phones but not desktops and laptops?
I find myself struggling to avoid falling into the trap of doing or avoiding-to-do things specifically so that they'll show up in the analytics in a certain way that looks nice, which feels like a weird "teaching to the test" but applied to my actual life. Some things I've since stopped tracking because I really disliked the meta-ness of it all.