What data on myself I collect and why (2020)

(I want to start with that story because I got myself into that mess by being a moron and figure that it's important you don't think this came from a place of intellect - it was 99% fear, trauma and trying to overcome the stupid defaults that got me into that mess in the first place.)

The first day that I got out of the hospital, I had to move back in with my Mom. I decided to go out for a walk and was in such a bad place that two little old ladies (one of whom had a walker) actually smoked me on the path back to my Mom's area. I was remarkably close to killing myself after that experience and in all truth, if it wasn't for my daughter (she was 18 months old at the time), I would not be here to write this. My GP referred me to a psychiatrist.

My psychiatrist is an amazing man and an incredible athlete. He suggested two tools that could help me with my cardio recovery but that have also been extraordinarily useful for my mental health. He recommended a Masimo fingertip oximeter and a training band by Polar.

The Polar training band is/was amazing. My first Polar was an H7 with an XXL band. Now I use an H10 with a small band. I'm not sure that I would choose the Polar again if I was starting out now - at the time, my psychiatrist suggested it because I could find a band that would fit in between my massive gut and the band of fat around my chest that used to lead coworkers to tease me about my bra size (I was assigned male at birth and continue to identify as male so that was not complimentary. It was funny at the time but quite sad to think about now.)

Program wise is where this story starts to get a little bit fucked up. Remember the first paragraph where I told you that I was writing code when my heart went to hell? Well, that really fucked me up and I quite literally could not write code or even sit in front of a computer. Computers have been my biggest love and practically my sole hobby since I was a 7 year old with a bad stutter. Suddenly, that was replaced with the most unbelievable fear whenever I would open up a text editor or even sit down at a keyboard.

Consequently, at first I started using mobile apps and ran everything through my iPhone. But as I started to recover and as my psych meds, therapy and other psychiatric help started to kick in a little more, my brain started to come back.

I have a marketing degree and really like data so as my brain started to come back, I started noticing correlations between cardiac metrics and purchasing activity.

That was strange. In a sense, my capitalism came back and I saw an opportunity. But, at the same time, I was a patient and doing cardiac rehab/support with ten older adults from my Mom's community. This sounds weird to anyone with a healthy heart but I'm a firm believer that whoever coined the term 'a broken heart' had had a cardiac intervention. It feels exactly the same and I had this group of older adults who cared enough about me to check in on me, come by and make me go for walks or even just sit and watch baseball with me. Consequently, I was never able to turn that opportunity into a business, but it gave me an idea about some code to write.

So, I started writing my own fitness tracker. At some point I might release it because it's been damned good for me. The first version is unbelievably sad and I have one hell of a lot of trouble reading that code over now. It was somewhere between a suicide note and a eulogy for my former self but that minimizes how brutally sad some of my variable and method names were. My favourite example is fatFuck.name = Greg.

That project turned into rehab and through the years I have kept adding to it. At first, it was purely about rehabbing my brain so I could write code again and maybe graduate to a point where I could comfortably refer to myself as user.name and heck, maybe even remove the word 'dead' from all my method names. But as it has grown, it has turned into a really good tool for teaching myself how to think in terms of healthy defaults.

Here is where stuff gets really fucked up sounding so if you've made it this far, good for you! I don't know if the app I wrote has actually helped me or if I hacked myself. I am deeply obsessive, to the point that when my psychiatrist did my OCD screening, I went home and rewrote my resume because the screening materials described my talents better than anything else I had ever written. When I build anything, I test it obsessively. In this case, to test it, I had to live it and 3.5 years later, I have a six pack, no love handles, two marathons down and an ultra in my imminent future.

Tracking nutrition against performance metrics yielded the most powerful results of all. At first, I started by writing down what I ate and then tracking my macros. I learned that what I thought was 2000 calories was actually closer to 4000 calories. As I started paying more attention to macros like protein and carbohydrates, I found a great feedback loop where eating healthier foods resulted in improved athletic performance. As my athleticism increased, I was able to work harder and then started tracking my moods against that data. That created another feedback loop where the better I ate, the harder I could exercise and the better I would feel.

At this point, 3.5 years later, I actually have healthy defaults. It sounds insane, but I genuinely have healthy defaults. The way I eat, the foods I eat and even the textures I enjoy have completely shifted around.

The only downside at this point is that I'm rather skeptical of most of the wearable industry. I've seen way too many metrics that make my capitalism tingle in funny ways. I know that I could enhance tracking with other tools but at this point, I'm pretty obsessed with reading their privacy policies and speaking as a patient instead of an entrepreneur, I don't think the marketing industry needs (or deserves) this type of data.

I feel like I've missed a lot. At some point, I'll put everything together into written form and do a better job of this reply. Sorry if this was disjointed!!

I'm still a significantly better runner and don't think I'll ever graduate up to a full Ironman. I have never even done an official triathlon, only self measured events. But having biking and swimming as part of my fitness package have been incredibly valuable. Swimming, for example, has done amazing things for my lung capacity (though again, it's not the swimming, it's my aversion to drowning). :)

Now I'll give you some raw numbers.

My resting heart rate has gone from a weekly average of over 90 to a weekly average of under 50. Simultaneously, my heart rate variability increased significantly. Raw numbers with HRV are a little misleading since it varies so much, but I am 43 years old. At the beginning of my odyssey (I had just turned 40), I had the HRV of the average 70 year old. Today, my HRV is way above average for a 20 year old.

Oxygen saturation is interesting. I went through a battle with covid in December and the nurse practitioner commented that she had never seen anyone as sick as me with such a high oxygen saturation level. When she asked me how I cared for myself, I told her that I had been working on speed all fall, doing sprints and running hills. Her reply was that she was going to start running up hills everyday on her way home from work.

My cardiologist talked me out of trying to actively track and monitor VO2 max. He explained that while it's a decent metric, I'm unlikely to be accurate enough at home to gain anything from it. Back when I used to get regular stress tests, I had a good number every 3, 6 and then 12 months. At my current fitness level, there isn't a particularly good reason to give me a stress test other than personal curiosity and the really fit, incredibly competitive health care worker who actually chirps me out after my stress tests. Next official marathon, I'm beating his ass by thirty minutes.

Mind if I share something that both my cardiologist and psychiatrist have told me? I'm not the typical cardiac patient. Instead, I'm deeply obsessive to the point that after my OCD screening, I went home to rewrite my resume as the screening did a far better job of explaining my talents than anything I had ever written. There is no particular need to take things as far as I have. I'm just 'special' and not necessarily in good ways. A lot of my fitness has come out of fear and if I could spare you anything, it would be to spare you that fear.

Heart problems are emotionally hard. If you want to talk to someone relatively young who has been there too, my email address is on my profile. We can keep everything anonymous if you'd like to feel more open to talk. I've been through a lot and friend, I've seriously got your back.

I did write the iphone app but it is quite simply the worst fucking mess of code I have ever written. I wasn't doing very well in the brain when I wrote it and holy hell, it needs a lot of love. I just wanted to be able to dump raw numbers and work with them on my own, but somewhere along the lines, I became really really really dumb. I'll fix it and let you have it if you want. You can do whatever the heck you want with it.

I was basically where you are, BMI and I and friend, I believe in you completely. However, you're going through PVCs and dealing with that anxiety so I'm going to share something a little dark. I really firmly believe that whoever coined the term 'a broken heart' to describe the loss of love had had a heart condition too. They are so close, emotionally and even physically.

Point being, you're going through a lot and if you need a friend to talk to, my email is in my profile. If you're anything like me, waking up in the middle of the night (4am was my hour) might be pretty rough. I'll give you my phone number and if you need, call me at 4am and we'll have a talk.

I believe in you so much that if you're interested, I'll hold off on qualifying for the Boston marathon and wait for you. Even if you don't want to run it with me, come to Boston. We'll celebrate how far we have both come.

Exercise is another area that has just become habitual. It started off as something I really hated but as I got more fit and started optimizing my nutrition, I got into an incredible feedback loop. The better I eat, the harder I can exercise, the more fun I have and the better I feel after. That creates a really strong feedback loop with my productivity - I can get more done now in six hours fit than I could accomplish in 10 hours fat.

I think it would be very hard for me to go back but it's worth trying out. I might give up the app for a few months and see what I can do without it. If I've merely swapped addictions, it's positive (for me) but my tracking regiment is bad.

As for your question, that has actually changed quite a bit over 3.5 years. In the beginning, it was purely about my heart because I didn't want to die. Now if I could only track one thing, it would be protein volume and amino acid profiles versus the intersection of raw athletic performance and recovery time. If I want to set a PB, I can usually hack that with nutrition. If I want to increase my weekly mileage, I can always hack that with nutrition. But at this point, I can only do one of the two (or else I poop my pants).

Edit - I forgot the most important question. My two favourite baseball teams are the New York Yankees and whoever is playing the Boston Red Sox. :)

Edit 2 - I wish that I was joking when I said 'or else I poop my pants' but I'm not.

Edit 3 - If I just wanted to create a baseline for how fit I am, I would likely use either burpees per minute or perhaps how far I can walk on my hands. Walking on my hands is a very good predictor of everything from swimming to running performance. Or maybe I'd use how many pukers I can bang out in two minutes? A single puker is one chin up followed by one burpee, both with perfect form. If I ever want to train people, I should come up with a better name than a puker but hey, I talk about pooping my pants on HN so what have I got to hide?? :)

I would suggest you invert the logic: design objective (ex: mental acuity), conclude on what things you want to track (ex: time slept, cups of coffee per day, time you drink it, errors in shell) and get the data accordingly, spending as necessary (ex: smart coffee maker? or maybe a smart plug that can let you infer when the machine is being used from the wattage drawn)

As for data tricks, L distance is a good one, yet not applicable for shell, as it's sensitive to the string length so you would need correction. Also, it's missing the essential metric: did the command work? Only the error return code will give you that.

TLDR: think about what you want, but before that think about why you want it. Collecting useless metrics is another form of data hoarding.

The Snowflakes minimum Is something my colleague got from them when we set up an account for a startup idea.

I've tried the same in snowflake, and I'm only using their smallest warehouse, and everything happens in seconds. Loading GBS of data is seconds, scanning GBs of data is seconds. Their smallest compute warehouse is still significantly beefier than the cpu you get with a puny lightsail with a shared vcpu and while it's disk io is shit (since it's reading from S3) the parallelism makes up for it.

In the end I think it's the pricing model that makes the difference. With snowflake you seamlessly launch a very powerful machine to run your query and only get billed for a minute, and with running postgres you need to run your machine all the time. Also hot storage (as opposed to s3) is also non trivially expensive. Haven't seen any way to get 200gb of ssd storage without spending that 25 dollar minimum that snowflake costs anyway. This is data I intend to not even update or query more than a few times a month, so I really don't want to pay for compute running all the time in its name.

https://altair-viz.github.io/