I put my whole life into a single database
howisfelix.today
howisfelix.today
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
/edit: quantified, not qualified
It was kinda interesting to see how many times I woke up, or track hours, but to be honest I realised after a few months that when my tracker said "You had good sleep", or "You had bad sleep" I was already aware - I woke up smiling, or grumpy depending on how I'd done.
I didn't ever look at the data and think "I want to go to bed now to catch up on the four hours I missed yesterday". I continued to have mostly consistent hours, but if I was doing something interesting I'd stay awake, and if I was tired I'd go to bed earlier naturally. The graphs and data wasn't providing anything of value, or encouraging me to change my behaviour in any significant way.
That was it, I got extremely annoyed by notifications so over time just disabled them. Also for some reason the heart rate monitor glitched a couple times, got alerts about my BPM at 180+ while I was sitting on the couch.
Eventually I just stopped using it and now sits in some drawer.
I was aware that alcohol affects your next day, even a little. That's because people always say that alcohol is bad for you (surprise surprise). I heard this, so you could say that I was aware. I generally thought about this as "a hangover is bad for you." and was somewhat dismissive of the "even a single drink has a bad effect" mantra.
I did some experimenting, and slowly realized that even a single drink can indeed have an impact on the next day. It's not a hangover, but an impact that I could feel nonetheless. I needed to do some light stats and a lot more journaling to build this awareness. I am now aware that I am aware.
If you could feel it why beed the stats?
There is a difference between knowing something, and believing it to be true.
I know that sometimes I feel good when I wake up. I know people say that drinking makes you feel not good when you wake up, sometimes.
It takes a bit of observation and some statistical sampling to connect those two together. Now I know it, and also believe it to be true.
Perhaps "aware" vs "aware that I am aware" was a bad way of phrasing it.
There’s also the motivation factor. I’m not sure of the total %, but I certainly did some exercising just to fill the daily goal. Nothing life-changing, but for the price of a cheapo apple watch se once every 5 years or so, more than worth it.
It’s not unlike simplistic time tracking on my iphone. I spent a lot of time on bullshit websites. Obviously I knew it was happening, but the sheer magnitude was surprising. It’s akin to acute pain letting you know there’s a health problem vs something brewing in the background that you are vaguely aware of, but have no motivation to truly care about - one is far more noticeable than the other
I've been wearing an Apple Watch for close to 10 years. I've tracked my weight as well along those years but nothing crazy like OP. The Apple watch tracked plenty.
I had some strange symptoms and two doctors insisted I had a weak heart and potential heart failure. This was shocking! Turns out I do have a really "weak" rhythm, but heart failure is when your heart is progressively getting worse in it's pumping. I don't even remember which metric he looked at in my Apple health - but basically my heart has always been this way. A doctor looking at a single data point might think I have abnormally low blood pressure/heart rate, but if I've had this for 10 years with no change, the medical assessment is very different - it means nothing. Sometimes boring data is exactly what you need. For this reason, I will probably always wear an Apple watch (or equivalent) moving forward.
Data can feel useless for 10 years until one day it becomes critical. The benefit is spiky and uneven.
I would wager that for most people, most data about themselves will be useless and not worth collecting.
Of course you can’t know what data will be useless or not, so unless the cost of collecting it is minimal or nil (wearing a smart watch, writing down your weight each day/week), it’s probably not worth it.
Spending hundreds of hours to build a solution to capture all data about yourself to find interesting patterns has a huge assumption baked into it: that there are interesting patterns to find.
I've gotten deep into weightlifting/bodybuilding over the past couple of years, and that's the kind of hobby where micro-optimizations and data tracking can have a pretty big impact on results (and sort of necessary, you can't fly blind with things like diet, especially)
E.g. I track and weigh everything I eat, take body measuraments on a weekly basis, Dexa scans every few months, etc - for me it's worth it because I know what I want to do with the data. If I didn't have a goal, all that tracking would clearly be overkill.
I've been weight lighting for ten years and initially tried to track things (down to how many reps I did of which exercise, with how much weight) and quickly came to the conclusion that is want worth it for me.
Tracking scale weight is difficult because shifts in water weight and hydration can swing the scale 5+ pounds in either direction without any change in body fat. So I pair scale weight with a 7-point skin caliper measurements taken on a weekly basis, along with waist circumference, in order to infer whether body fat is trending up or down. And also take weekly progress photos of 6 angles/poses with consistent lighting, which I share with a coach.
And then you pair that with weighing and logging everything you eat, and you can make small adjustments to your meal plan on a monthly basis to try to stay in that 200-300 calorie per day surplus for as long as possible. (Although most bodybuilding coaches adjust diet based purely on how your physique is changing in weekly check-in photos without the need for measurements, but I like extra data)
> down to how many reps I did of which exercise, with how much weight)
I also do this. Track every exercise, every weight, number of reps. It's necessary for knowing whether you're progressively overloading over long periods of time. Progressive overload becomes harder to measure once you're past newbie gains because you can't increase weight every week, so some weeks the goal is just to squeeze out an extra couple of reps. Which adds up over time
This is obviously excessive for 99% of people. But I enjoy doing it as a hobby. I would absolutely not recommend this level of tracking for health reasons (not necessary) - I find enjoyment in the process.
I still suffer intermittent stomach aches, especially in the early hours of the morning, and had a terrible time trying to decide if they were getting better or worse over time.
Our narrative voice is awful at detecting long term trends and tends to overcompensate for particularly good or bad patches so it was impossible for me to judge and I started keeping records of how bad the aches were each day.
Long story short, the average severity was mostly decreasing over time and the average time between bad aches was slowly increasing but it would have been impossible to tell if this was happening without keeping detailed records because it wasn't consistent - some months were much worse than others and completely skewed my perception of long term trends.
While most people hopefully won't ever need to do something like this, it did make me realise just how bad we are at picking up on long term trends so I can definitely see keeping daily records of, for instance, average daily happiness being eye-opening.
Proactively capturing and tracking everything you can to prep for any future is too much work that would really steal your time from, you know, actually living a life.
Not sure if in your case the data was critical, since the doctor likely would have just had you wear a monitor for a while after to come to the same conclusion.
The lesson, I think, is everything is relative. Even a dashboard with flawed data that is "consistent" can highlight anomalies. And often, that's all you really need out of them. (Or the lack of anomaly)
But I see people start min-maxing these numbers as a replacement for big picture health goals.
From the outside, I see someone spending a lot of time focusing on numbers while they are actually regularly stressed, who doesn’t get good sleep, and has somewhat bare minimum exercise.
Collecting data is great but don’t sink so much effort into it until you have a problem.
But, for anyone who does, there is another 1000 who do not when something hits them: many illnesses develop gradually, and all of our tests (thousands of blood tests, scans and imaging tech...) would benefit from having historical data when we were "ok".
Similarly, you probably did not have more data than what Apple provided to help narrow the problem you still had, right?
And if everyone was put under so many tests, we'd actually be "solving" a bunch of non-issues for people over-reacting to small deviations from "normal" range.
Apple watch helps you with a few parameters — not to be discounted — but I don't really see it as a counter.
I did something similar to pull data from my Garmin watch. This meant writing all manner of code to pull data out of FIT files (interesting and often infuriating self-describing file format), coming up with schemas to hold that data to make it queryable, adding visualisations, performing analysis, pattern matching, etc.
The end result is nothing really useful, I had a bunch of scripts that semi-automated some jobs that would have taken 1 minute to do manually and only ever needed to be done a max of five times a day, but I learned a load of things along the way. Often these were useful lessons that can be applied to many other things when developing software.
In a similar vein I've gone to lots of trouble to build a cooling system for my homelab rack (ESP32 to control PWM fans, Dallas 1-wire for reading temp/humidity, exposing measurements as metrics for scraping/observability, designing things to deal with the different voltages involved, etc). I could have just gone and bought an off-the-shelf solution from AC Infinity and installed it in minutes but where would the fun in that be.
I do think it's not worth spending a whole lot of time on, though - hence why the first thing I did was add that mechanism to have Claude build it for me, with me mostly glancing at a plan and saying yes/no. It's the perfect thing to vibe-code - if it breaks, I revert a commit and it doesn't matter because nobody depends on it but me.
Why? Because those individuals tend to spin something up, tell everyone about it (online, and offline) and then stop doing it few days later.
The result then ends up being a false signal for others in the same boat. People who read it, feel a spark of recognition ("someone like me actually figured this out"), and then invest real time, energy, maybe money, into replicating something the author themselves quietly abandoned two weeks later.
Just a small heads up from someone who used to get burned in the past :)
I've absolutely not figured it out, but I now have an agent throwing stuff at the wall (with guidance from read access to e.g. my journal and a few other data sources) to figure it out for me, and it's gotten steadily better.
That's definitely me (most recent ones: using engineering notebook techniques but for my own life, and WOOP method), but I recognize that feeling like I've found THE solution when I'm only a few days into it, so I tend to wait and see, or if I tell someone I say "...but ask me again in a week or a month if I'm still doing it." (At least with the engineering notebook, I can still go back and use it to remember what steps and settings I used in KiCad or use WOOP on a new goal at any time. So it's not a total loss.)
I will say one thing that I have stuck with and is pretty useful is a morning checking and an evening checklist. I'm currently using a paper version with the days of March in the columns and the checklists in the rows, and X them off as I go. A slash for the one I'm doing now/next and X when it's done. Leave it blank (or write N) if I choose to skip it. As a back-up, when I can't get around to make a paper version (I'm planning to type in the steps in a spreadsheet so I can just revise and print it each month) I keep the lists in two Google Keep checklists. Those are great because you can reset the checklist each day for reuse, and you can drag to reorder it as you edit it, and you can indent one level to organize it a bit. The disadvantage is I might get distracted by notifications and stuff on my phone.
What's the difference here between assuming and saying 'ADHD-like' instead?
Above all, it's just interesting. I enjoy reading about the day-by-day progression of a crush or my brutally honest feelings about a trip that produced stunning pictures. It weaves nuance into my history.
A good thing.
On the other hand, if you are sick like me, charting your long term heath data from doctors visits and photographing skin issues can lead to great discoveries. I have been diagnosed with Erythrocytosis and a susceptibility to mycobacteria infection which caused pulmonary nodules and skin lesions. Only after showing my data collection to my doctor. Since I have mental illness they constantly over looked my physical issues so I needed hard data to convince them of my ideas.
For those curious, I have an minor IL12B deficiency and a partial immune deficiency leading to mildly elevated levels of DexoyATP which is partially corrected with zinc supplements.
We already mostly know what makes people happy/healthy: personal connections, physical activity, healthy diet and some sort of purpose/goal in life that goes beyond day-to-day activities. The problem is that these things generally require (hard) work and can be unpleasant sometimes, so humans do what humans do and spend unreasonable amounts of time doing the more pleasant things such as reading and gathering info rather than applying these and what they already know. (That's not to say that a project like this can't be fun or lead to insights, especially across longer time spans, but i feel like all of the questions in the first paragraph have fairly obvious answers if you know yourself at all, that don't require extensive tracking of stats to get)
Quantified Self, at least in the intended form, is focused on testing specific theories, not on collecting large amounts of data and trying to find something interesting in them.
See, for example:
https://gwern.net/zeo/zeo#what-qs-is-not-just-data-gathering
I agree with this but minimizing the cost changes the ROI.
Personally, I've discovered useful insights tracking various life metrics. But I also found quickly diminishing returns after a few weeks or months -- if an association isn't obvious within that timeframe it's either too much effort to isolate or too slow or small to matter.
At various points I've tracked calories, macronutrients, weight, allergens, supplements, sleep, exercise volume, exercise timing, nighttime screen use, spending/budget, air quality, and mood. Now I know what kind of cooking wrecks the air quality in my house, what foods I don't digest well, what various protein/carb/fat ratios look like on my plate, how much effort it takes to improve fitness, that exercise in the morning or early afternoon improves my sleep while exercise in the evening harms it, and that any alcohol or caffeine wreck my sleep while screens at night have no measurable effect. But once I understand the associations I can alter my behavior and move on.
> The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
I would agree that continuing to track metrics every day long after they've stopped yielding new insights is often compulsive behavior. But I think that's an argument for time-boxing experiments, not necessarily avoiding them altogether.
I liked doing similar things in the past. There's no anxiety in the equation, just pure curiosity. How many times have I done a thing a month/year? I was always curious about stuff like this, much like the OP. There's also the hacker spirit in play - designing the apps for tracking stuff.
Who knows how many hours I spent scanning nutritional facts on the backs of boxes, estimating amounts of liquids, and even tracking sips of water. And weighing myself! Thank goodness I used a "smart" scale at least, and I didn't have to worry about carefully inputting my weight to an app each time.
But the whole project was an exercise in perfectionism. "I have to remember this sandwich and log it the next time I'm alone" made me anxious, but once I logged it, I felt a sense of completion. The database and my personal history are now at nirvana. Everything is complete.
All for me to learn things every human alive knows today: eat more food and you'll gain weight; eat less food and you'll lose weight. Yes, I can now tell you the exact average difference in calories I'll eat, statistically speaking, on a day that I have adderall in the morning vs a day that I don't. Yes, there's a similar (but much smaller) difference in average calories per day if I have caffeine in the morning as well. And I can tell you that I generally eat an additional 200-400 calories per day on a Fri-Sun than on a Mon-Thu. Wow, groundbreaking.
I've always had a lot of water, but matching foodanddrink.csv to my HealthKit data showed that I have more water on days that I walk more steps. Mildly interesting to see it written out for me? I guess. But was it worth cataloguing every cup of water? Absolutely not.
Was any of [gestures broadly at me pulling out my phone and cataloguing each item I ate] necessary to learn that? Of course not. It gave me a chance to look back at the database and say "Wowee! I did that! Every day for a year, wow, I'm so cool!" and not much else.
Trying to extrapolate this conclusion to the entire "quantified self" movement is not correct. The issue is the time cost, not the act itself. If a trusted company came along (as if...) that sucked up this much data to allow you to answer these questions with minimal effort, I'm sure this would be a different story.
Anything at the fringes of tech with no tried and true solution requires hundreds of hours of effort. The author's conclusions are also personal, there are other styles of living and conclusions to be drawn that change the calculus on whether to do it or not.
* Hardware companies went out of business, stopped supporting devices, etc. It became obvious that there was no long term commitment to make good quality hardware that lasted a long time.
* Many devices and/or data collection was consolidated big, data hungry companies Google and Apple. Competitors have similar anti-consumer uses of data. I don't want any of these companies to have my data.
* Related to the last one, limited to no offline or local only data collection.
It is very hard to gather most of this data with off the shelf hardware and keep your data private.
Felix's statement isn't a condemnation of quantified self. I think that sometimes, when you're applying algorithms that aren't well-studied, you get pointless or bogus data. I feel much this way about sleep tracking algorithms from the likes of Apple and Whoop. Vitals, too, although it seems remarkably good at detecting when I'm about to get sick.
As a person in the older demographic of HN, having an Apple Watch quietly collect useful data over years and store it in a database has been immensely useful in helping resolve medical issues.
I used to really be into QS, and if I'm training for a marathon, I'm still studying graphs and drawing conclusions. I'm still into QS, but now I just silently log data to Apple Health and use it for medical histories or to identify certain trends (vo2max, cardio fitness, blood pressure, etc) a couple times a year.
Yes it is! And if you control everything, you won‘t make mistakes.
As someone who has dealt with OCD and perfectionism, I think that could be the case for a lot of people. And the urge to obsessively track everything can be debilitating.
Paris accord says 1.5t per person per year, from all activities, Felix's flying alonre is ~10-15x current European yearly per person emissions and ~50-75x those compatible with +1.5C.
What does this have to do with Felix?
Unrelated link: https://xcancel.com/Ryanair/status/776292730179682304
I get that you may have to see family abroad or maybe indulge for a holiday, but this is "I'm using an airplane to commute" kind of level.
And here I am trying to book my train tickets to go to London instead of flying even though it costs three times as much just to avoid a few kg of CO2 (among other things), it's making me angry.
Reminds me of the soggy straw memes floating around now. I've been having those why bother? thoughts as well.
On the price, the very annoying thing is that fuel for planes is not taxed! Changing this would require quite some effort (falls under some specific laws, that are old and nobody wants to touch, etc.) but I think everybody should just ask "honest tax on fuels!" as this will make less people say (or thin) "but climate change is a hoax". Planes are just unfair competition to other transport due to taxes!
An alternate approach that would be seen as consumer and business-friendly would be subsidizing companies with a certain level of fuel efficiency per passenger mile, targeted above current levels.
That's 14 round trip flights. A little over once a month. Not at all surprising if once a month. Or divorced and kid is in another state.
If we're generous and assume each trip is actually 2 flights (i.e. connecting flights), then it's 7 per year.
Yeah, more than typical. But not insane.
Typo: Should have been "Not at all surprising if for work".
We're adults, we can keep many things in our minds at one time: We should all reduce flying. Regular working people should not be shamed for taking a holiday and flying there. The most frequent fliers for work should make a personal effort to reduce their flying. And companies, conferences, etc. should work much harder to facilitate remote participation and reduce stigma around it, as well as encouraging other modes of travel. Governments should improve alternative solutions such as rail and high-speed rail.
Some — eg. those who've moved for work — probably fly 10-15x the average, and 10-15x of people fly 1/100th of average. Flying cost and availability correlate with average needs, but that presupposes that some people have higher needs altogether.
Shaming individuals doesn't seem to be productive or helpful.
Air travel works for people if the benefits outweigh the costs. The only thing that changes behavior is to change the costs.
And even if costs were 10x there are still plenty of people who will fly tons, because it would still be economically productive. There are always going to be people who fly 10x more than others, because certain jobs and roles simply require it.
I don't see how much support from history for that viewpoint. Some examples of positive societal change driven in part by shaming individuals: drink-driving, civil rights, sexual harassment, automobile safety, the slave trade, McCarthyism.
Automobile safety in my life has only changed after fines. Sexual harrassment still happens and doesn't seem to be helped by shaming someone as much as firing them. Though we often don't have the guts or legal backing to publically shame someone.
This hasn’t been a good few years for your examples.
Let's raise the tax on an activity according to its negative side effects, while pointing out individuals that do a lot of it and dont take personal responsibility.
I don't know this guy's personal life, but the people I know who fly tons fit into this profile. E.g. the wife can't move because she's a tenured professor at her university, and he's got to be at both offices regularly. He's best qualified to run the company/companies, and he's not going to get divorced to reduce his CO2 emissions.
What exactly is the solution you propose? What personal responsibility do you expect them to take? You think he should get divorced? Only see his wife and kids four times a year? Have his company/companies suffer because he can't be there in person? Quit his jobs?
And let's be clear, there are lots of jobs that require tons of air travel. If you're a highly specialized repair technician for certain equipment, all you do is constantly fly around the world fixing equipment wherever it is. If you're a CEO of a multinational company, you're constantly flying around to different offices. Are you looking for "personal responsibility" here too? How?
I'm sorry, I don't want HN to to be the place where we get into a fight over the mildest inconvenience for people who are already living extravagant lifestyles.
I suggested raising taxes in the first place.
What I'm opposed to is some hand-wavy demand to "take personal responsibility" without suggesting exactly what they're supposed to do and whether any tradeoffs involved are reasonable.
And please don't call people names. You can write comments here without calling other people "keyboard warriors". Nor is it helpful to try to shut down some viewpoint by claiming that somebody doesn't need any extra support.
And I think most people would consider not seeing their family more than e.g. four times a year more than just the "mildest inconvenience".
Knowing this completely changes the tone, I’d missed that you were that same commenter too.
I don't see any shaming. It was all matter of fact, free from judgement.
Do you think that comparing someone's CO2 emissions with the average and pointing out that it is much higher is value-free, just a totally neutral observation for no reason? That the commenter is fine with it? Or even that it's a good thing?
But I'll also respond to your questions: my purpose is to show that your claim that the original comment was "free from judgement" is wrong. I'm not neutral, I'm attempting to show that your claim is obviously false, that it's not plausible at all. Of course I'm trying to judge a comment that seems wrong.
So now that I've replied honestly to your questions, will you reply honestly to mine? Repeating:
> So you think the commenter was neutral? No judgment? Again, what was the purpose of the comment then?
Because if the purpose wasn't to shame the person for their carbon footprint, I can't imagine what else it possibly could have been.
But you see: not one comment here is neutral. It would be silly to expect a comment to be neutral, such a comment wouldn't be written in the first place. I think the original comment expressed the point while staying as neutral as possible.
> So you think the commenter was neutral?
Yes, it stated some facts.
> No judgment?
Yes, it contained no explicit value judgement. Any value judgement we bring into it is our own.
> Again, what was the purpose of the comment then?
How would I know the purpose of someone else's comments? I don't even really know what my purpose is debating here with you. I certainly don't see myself persuading you of anything :)
I don't know what definitions of neutral or value judgment you're using, but I hope you can use this as a learning opportunity. The original comment has the obvious implicit judgment that a greater CO2 footprint is a bad thing. This is shared context. It is so obvious it doesn't need to be explicitly stated, any more than "murdering people is bad". The purpose of the comment is clearly to shame the person for having such a high carbon footprint, otherwise there's no purpose in bringing it up. I don't know what your purpose was in trying to deny that. But if you genuinely didn't understand before, I hope now you do, and that this has been helpful in improving your reading comprehension or understanding of shared/implicit context.
No it did not contain any judgement. That is your reading and not the comment.
> This is shared context.
The world is a large place and different people share different context.
> The purpose of the comment is clearly to shame the person for having such a high carbon footprint, otherwise there's no purpose in bringing it up.
Informing perhaps?
I wonder what your gripe is: Do you think it's inappropriate to mention the CO2 at all? Or do you think that the poster should have said it in a different way? In what way?
That makes as much sense as saying that the comment being written in English is just my "reading" of it, and that in a different context all the words could have entirely different meanings. That is technically true, but also completely useless when you're engaged in a real-life conversation with the rich context of the year 2026 on planet Earth... I tried to explain to you the shared context in case you were somehow ignorant of it. If you insist on denying that, then you're going to have a very hard time communicating with others, as you seem to be having with this conversation. Good luck.
> Shaming individuals doesn't seem to be productive or helpful.
First, none of us have any power to "tax it more" so this is a dead end of discussion. Second, people have agency and we can hold them accountable socially for negative actions even if they are abiding by the current laws (or tax regime). This happens all the time, because laws don't fully align with morality in a culture. Suggesting that we should leave such things to the sole discretion of the economy and taxes describes a strange unhuman-like society that we don't live in.
You probably have more power over that than changing a whole population's behavior.
I'd wager you probably have more power over that than changing just one person's behavior (Felix).
> Second, people have agency and we can hold them accountable socially for negative actions even if they are abiding by the current laws (or tax regime).
And people are holding you accountable for the shaming. This leads primarily to polarization, not change. Polarized folks are more likely to increase the negative behavior.
Well, the choice is: either we leave it to economy and taxes which work, or we'll rely on shaming which doesn't work. To put it another way - it depends if you want to actually fix the problem or just want to have a moral high ground and don't really care about solving the problem.
That said there are probably some work-arounds, tax free twice a year, tax rebate or some-such.
No, it's the other way around. If flying is bad then poor people absolutely have to be priced out of it. I know that it sounds absolutely awful, but unfortunately taxing just business and first class won't do much for reducing emissions if flying will be affordable for and used by billions of people.
Of the very few "f*cks" I can give in my life, I prefer to spend mine as I choose rather than being scolded for not giving mine to the pressing issues that others deem important.
If you truly care about the planet, don't have children.
At the very least don't brag about not giving a crap.
That's a fallacy; people care about the planet precisely because of children. I don't care about the planet for its own sake; I care because of the humans who inhabit it and their future lives.
Also, humanity spent 100,000 years without flying around the globe, and I doubt they were all living hermit martyr lives.
The fact that I happen to care about other things more than this specific flavor of global catastrophe is morally OK.
Shame is not necessary, but callous indifference is not acceptable. There is a middle ground where you treat it like a habit to improve, like a step count.
I'm also not going to take shorter showers when people are farming in a desert and shipping the crops to China.
You might think this makes me a terrible person. That's probably good. Because it will help people understand what we're up against and what needs to happen to actually solve the problem.
"Take less flights" isn't the solution.
You may wonder how this is consistent with my propensity to recycle and follow traffic laws and not do crimes and other socially beneficial minor things to which a comparison could be made to CO2 output: because those all have a greater benefit:cost ratio than flying and eating meat, by far. I am including the benefits of living in a high-trust society in that analysis.
Instead we're razing the forests and bragging about our indifference.
What's key is be able to visualize metrics easily on the data and frictionless data entry, I've got a decent setup with iPhone Action + Obsidian + QuickAdd scripts on Obsidian Sync (mobile + laptop). for visualization I use Obsidian Bases and Obsidian notes that run Dataview code blocks and Chart.js, couldn't be happier.
I could track things that are not interesting to reflect on like vitamin D supplementation for accountability but I've never bothered, especially if it's taken ~daily.
I have had people tell me they were "manic". Then I showed them videos I took when I was manic and they see what I mean when I tell them they are not manic.
We have come to a place where we do not want even normal fluctuation in mood, and that is a illness of its own, but it is a cultural illness.
I am just trying to save you time and escape the cycle of "optimizations" which is where all this data logging leads.
It turns out that our memory storage device uses a very lossy form of compression. Memories get simplified and distorted over time. Heck, I can't even remember when something started hurting, so how should I notice a year-long pattern of thinking around a certain topic?
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
“This entire month I’ve been feeling good, I want to pinpoint why,” or “it’s clear since stressor X entered my life, my affect is lower; how can I resolve this?”
These long term trends are harder for me to track without data. It might be easy for others, but not me!
https://apps.apple.com/us/app/reflect-track-anything/id64638...
Every time I try to seriously track metrics of my life, the excitement of the insight gets worn away by the friction of recording and managing. I expect LLMs can help reduce the cost of this by an order of magnitude but then, as you mention, the question is, what do you do / change / learn because of the data?
I recently started tracking nutrition macros with an iOS app MacroFactor which I really like. This is the first time taking my weight doesn't feel like a IDK SHRUG moment and I can actually map my food intake to my weight.
Finances is probably the other highly actionable data source that is such high friction to manage (downloading CSVs, OFXs, monthly...) that it has always been a false start for me. I finally wrote a service to talk to Plaid directly and I successfully used it to categorize my business expenses at tax time. I finally have programmatic access to my bank account data!
You conclusion is definitely a cautionary take: > the main conclusion is that it is not worth building your own solution, and investing this much time.
But, perhaps a subset of that data you find useful.
I kept a rough log of my sleep and mood for about a year with no specific goal. Mostly forgot about it. Then I had a weirdly bad few months and went back to look — turns out there was a pretty clear pattern I would've never noticed in the moment.
Maybe the framing of "was it worth it" is the wrong question. It's less like an investment with a return and more like keeping receipts. Useless 99% of the time, then suddenly you really need one.
Forgive me for I am being sceptical. The might be some insight here I have not considered, but I'd feel a lot more comfortable if it was all self-hosted / self-collected data.
I had the same epiphany as you days after acquiring a CO2 monitor. Most people notice poor indoor air quality from proxies such as humidity and temperature. AC (without ventilation) eliminates these and tricks our senses very effectively, giving us cool and fresh feeling indoor spaces full of CO2 and devoid of oxygen.
It's possible the friction could be reduced here by having some kind of Generative AI try to help capture data, but then you'd have to verify that it was being done correctly... honestly, I think it's simply not practical for most people to do this.
Real life is messy. How much time you want to spend on recordkeeping to make it seem less messy or make you feel like you have more control is up to you. But sometimes it's better to embrace the mess and let go of control, in my opinion. Chances are, no one's going to care about whatever you do here in 100 years. YMMV as always.
https://edwardbetts.com/agenda/trip/past https://edwardbetts.com/agenda/trip/stats
Most of the things that really hinder my productiveness and happiness are the same things that everyone tells you about; sleep, diet, sunlight, not being stuck indoors all day, socializing. And most of these can be improved by making changes to my habits.
Other than that, is that I think that it's important to get into the habit of self reflection. Having a feedback loop on my own life has helped me find out what works and what doesn't. It's too easy to just go through the days slogging through and not making any changes because you don't even realize what's going on.
[0] https://en.wikipedia.org/wiki/Robert_Shields_%28diarist%29
The database is the data, collected into a coherent structure.
The database management system is (optionally) software. :)
Another tangent, recently bought a paper shredder, started shredding through boxes of mail I have kept since they have personal info like cc statements but on the same idea of moving on/reducing stuff I'm hoarding whether it's data or physical.
Did you think of building some proactive AI tools to make use of all this centralized data?
380k datapoints sounds incredible but I imagine the real challenge is turning that into decisions that actually change behavior
I went through a similar process and came to the same conclusion, although with a slightly different twist. The project became the point and I almost stopped noticing the data towards the end.
I wonder why? Also, perhaps worthwhile putting this at the very top of the page, rather than the very bottom.
Never heard it phrased like this, and wondering if this phrasing explains the rest of the project as well :)
This is why I moved to Tokyo. Even if I want to avoid exercising I still take many steps
https://apps.apple.com/us/app/reflect-track-anything/id64638...
Don’t ask how I know…
I know this is the type of person i would not like.