Show HN: Interface for all digital aspects of my life

Ah I see, so your approach is using databases as an 'intermediate' layer, rather than the main source of 'truth'. My main objection to databases is the fricton with the maintenance and the speed of prototyping, but if it's automatic and used as a cache, I agree it's reasonable. Although I'd still be worried about normalising something wrong, but this is my personal sort of anxiety :)

I've got something similar [0]: if you have a sequence of NamedTuples/dataclasses, you can get a database 'for free'.

[0] https://beepb00p.xyz/unnecessary-db.html#cachew_cachew

It downloads your data from social media (or other online sites) into a single sqlite database. It can even draw relations between various accounts/people. I use it to back up my Google Photos.

(Shakes fist at the RTM API which would return one and many child tasks in different ways, so it took me an extra half-hour to write the convoluted Jq query.)

> Can I search all my conversation with John across 3 messaging apps, email, SMS and phone?

These are totally pain points for me / questions I'd love to have some answers for... Anyone tried to get the historic iOS location data to answer the first question?

YNAB's method is excellent for this. TLDR: you record your expenses for some time, and then you begin planning them ahead for each month semi-automatically, putting in money for costly purchases in advance and leaving some for when shit happens.

I've been kinda looking for an OSS/self-hosted implementation of the method, but none seem inviting so far.

(G2A seems to have a Steam key for the old version of YNAB, under ‘You Need A Budget’—which version could sync via Dropbox, without its own cloud service. But the key costs quite a bit more than what I've paid for the app back then.)

> Which parts of my city have I yet to explore?

I'm using desktop QGIS to mark my trips, while OsmAnd on the phone shows the exported GPX on the go (among other pleasant features). In theory I can use OsmAnd itself to record the trips with GPS, or any of the other dozen apps for that, but personally I don't believe that Google won't suck the location up regardless of my settings.

Hm. I actually don't agree with that sentiment as written. It's not time wasted if you enjoy it and have something to show for it? That's not it either.

Maybe this. I view the work we're discussing like a programmer's equivalent of woodworking in your garage. Sure, it's not necessary. But it's creative, fulfilling, and enjoyable. There's something immensely satisfying about using something you made with your own two hands, even if it's not perfect. As long as all these things remain true, I wouldn't call it time wasted.

To me your critism reads something like, "if you wanted to canoe, why did you spend a year making your own? you could have bought one and spent that year doing what you really wanted to do!" Well, I wanted do that, but I also enjoyed making something to do that too. I feel my life is better for having made it.

yep, same

- One is along the lines of post-privacy:

Lots of the stuff that I'm collecting one can find online anyway if they deliberately search for it. Tweets/reddit comments/instagram photos, etc. Some of it isn't public, let's say, Amazon purchases. But would I mind sharing it? Maybe not, who cares what I buy? Some ML algorithm that would show me ads? Whatever, I don't click on them anyway.

The more stuff I don't mind to be public in the first place, the less is the chance someone can use it against me. But I understand it's ultimately something very personal.

- Second is that I agree that the security professionals would handle the data much better than me.

I don't even mind Google keeping my data, if only it was easier for me to access it when I need it! So generally I would happily pay professionals for a service to keep my data safe, being able to access it and integrate together, and for the infrastructure around it.

Same way I'm using a bank to keep my money safe. But such infrastructure for data simply doesn't exist at the moment, because there is no demand for it from people.

And silos only benefit the companies that keep this data, so the change isn't going to be come from them. E.g. if you can easily access your tweets through a nice and fast local app, why would you even visit bloated twitter.com, which is trying to 'entertain' and 'engage' you?

I'm hoping that with projects like this, I can inspire people to think differently about their services and tools, and demand (from professionals, i.e. engineers/designers/product managers) for better means of using their own data.

Well this is already happening. Google and facebook for example earns basically all of their money from this. They replaced advertisment middle men because they can track and quantize human behaviours and identities. They do it because they are able to aggregate the data and provide API for marketing people.

This is a tool that brings it closer to you as the creator of the data. it doesn't do anything new just exposes what's going on.

If you are able to open pages through EU check out data processing partners from GDPR consent you have to accept. Lists on every professionally built services are long. There are literally thousands of companies accessing, aggregating, processing and reselling the data and their analysis.

The tools you are afraid of already exist and are commercially available.

I think most people would want amnesia for certain memories like traumatic events. E.g. a romantic interest rejected you, or football quarterback throws an interception and needs to forget it to fully concentrate on the next play.

For most things, I wish I had more digital documentation of my past life. There were places I visited once that I wish I had photos for. There were people that were important in my life but I forgot their names and never wrote them down. I started keep a daily journal for the last 10 years and it's been so useful that I wish I had been doing that since I was child. If I had a git repo for everything single piece of code I wrote since I was a kid, that would be cool for me to revisit.

Back to your point, another type of purge that's useful are open projects that just nag you. For example, I had a "home automation" project on my todo list for years. In addition to taking space in brain, there were half-assembled open boxes of electronics on the shelf for years as physical reminders of that unfinished task. I finally decided to be realistic about my enthusiasm for that project and just abandon it. I sold the electronics and I feel better now. If I later decide to tackle it again, there will be newer and better technology anyway.

I also had a bookshelf of foreign languages for years that I thought I'd learn for foreign travel. After the COVID-19 crisis, I decided to give them away since I won't be traveling overseas for years. Now their absence no longer remind me of things I never got around to.

Purging bad emotions and unrealistic projects -- yes. Purging life data -- generally no.

I never really look back through any of it but when I do it’s quite nostalgic.

I think there is a happy medium and you can achieve it by keeping your inbox inside your trashcan.

This is not my idea and I have heard of it, in various forms, from different corners.

The idea is that all of your "important" papers get placed, as you receive them, into your larger (deeper) than normal trash can. I use a deep tray for this. It is basically a trash can that takes 6 or 8 months to fill.

When you fill the trashcan, you remove the bottom 1/4 of it and actually throw it away - since it turns out it wasn't important or necessary or actionable.

You can (and I have) created digital metaphors for this:

For instance, I dump my profiles directory, with all of my browsing history and bookmarks and so on, every month or so and then reset my browser. Then I delete those tarballs later when it's clear they were not important.

The key is this:

You can retain maximal optionality while still minimizing processing and sorting time. Just chuck it all into the "trash" bin and if it's really important you always now where to look.

So I chuck it in an archive. Gradually expanding storage at low prices means never having to think about deletion.

I don't use any of the systems for sharing tabs or history across browsers, because each has a different set of almost abandoned tabs. Sometimes "tab bankruptcy" is forced on me by software failure, and it's frustrating.

Notes, not really, there's a ton of them in Keep so I look up some stuff from time to time.

I have a massive collection in OneNote notebooks, and once a year it seems I check them and remember some good stuff, but mostly just how dumb I was before :D

I've been working on something quite similar to this project (in Rust), and I had the same need.

I started tinkering with a library + DSL that describes data fetching and transformation, where all data is transformed into schema.org types.

The DSL can then be either interpreted or client code generated for various languages.

This would potentially make such a API collection library much more feasible.

Sadly such a DSL turns out to be more of a full programming language if all requirements are covered.

That would be so useful, as also kinda dangerous. It would be also a damn amount of work, just for managing it. Maybe building some general framework and API and let people build it decentral?

[0] https://beepb00p.xyz/hpi.html#usecases

[1] https://github.com/karlicoss/promnesia#demos

In the long run, I'm not sure how sustainable it is, since there are many different data sources and ways of representing them. This is also particularly difficult when you don't use the service, e.g. as a maintaner I wouldn't have any means of testing Traktv.

So far I've kept all HPI modules close, because a monolith is easier for prototyping and refactoring. But my fear is the fate of oh-my-zsh, or spacemacs, which are overwhelmed by the pull requests.

Ideally I think it should be a simple core, only containing the common utility functions, extraction helpers, error handling, caching, logging, that sort of thing; and make the rest third-party.

It's possible to achieve this in Python, thanks to the namespace packages. The only problem I see is managing these small individual packages, and declaring dependencies between them. This is possible with PIP and setuptools, but there is certain overhead involved, I feel like this step is ought to be simpler, especially to people who don't want to fully dive into Python.

[0] https://github.com/karlicoss/HPI/blob/master/doc/DEVELOPMENT...

Probably in combination with org-mode http://wikemacs.org/wiki/Blogging

Yep, totally agree it's hard to incorporate the future, especially when it's in free-form like org-mode notes. Past data is much easier becuase it's at least somewhat structured and easy for the computer.

I guess my take on this is that subjective metrics, like the one people use to define 'success', 'growth', etc., are orders of magnitude harder to grasp than objective metrics (e.g. hard data, like sleep or excercise logs). Yet, we don't even have good means of incorporating the hard data in our lives, so I chose to start with an easier problem.

Yeah you're right, my future planning is all in my org-mode notes, I reflect on it with my brain, but so far not sure how I can use software (apart from organizer software) to aid me with it. It would certainly be an interesting area to explore.

Note that it's meant to be running on your own computer, and it's using the filesystem to access the data, no network interactions involved. Ultimately it's about how you're protecting the data on your disk and whether you trust youself with it. Of course, my code has to be trusted too, but it's at least possible to run it in a sandbox/use Docker, etc.

The modules run untrusted Python code, so if you keep your token on the disk, they can potentially steal it token too, unless you use some elaborate authentication system.

Next step: yep, I throught about it a bit [0], agree that it would be great as a service-agnostic way of accessing the data, the same way we can see it with our eyes.

But figured I should start with strictly easier problems for now, ones that I can solve at least to some extent. For now my goal is simply enjoying using it, making the setup simpler (to make it accessible to more people), making sure it's robust to the changes, flexible enough, etc. I want to inspire people to own their data. In many ways, I'm advocating an approach to owning it and working with it, rather than a specific set of tools, formats, etc. It would be great to have more people experimenting in the same area and trying to loosely integrate with existing efforts.

Optimizer: to some extents, yes! Not sure about 'general happiness', but I'd happily let the computer dictate me what to eat, how to exercise and how to sleep. Maintenance is probably one of the most annoying things in my life.

[0] https://beepb00p.xyz/pkm-search.html#future