Show HN: System for Private Data Management | Hacker News Reader

Show HN: System for Private Data Management

github.com

132 pointsdariuszW44 comments

I love your idea. This "life management system" space has tons of potential. It needs a unique combination of great data model, consistent syncing and flawless (responsive and attractive) user experience. I think in 5 years you'll see the next unicorn here.

I've used a few variants for "personal CRM / personal tracking"

  (a) "Unix" approach: csv files & encrypted text files (e.g. credentials) with gpg. Searching using grep.  Edit with Vim.  
  PROS:  compatibility, CLI tools.  cons: gets out of sync across devices, poor experience on mobile. 
  (b) tiddlywiki:  pros: works on desktops, good search, cons: poor sync to mobile, poor ux on mobile. 
  (c) "Google Ecosystem" e.g. Keep (notes), Contacts, Keep (todos), Calendar – Pros: syncs everywhere, cons: apps are slow and limited; also lock in
  (d) "Apple Ecosystem" – Notes, Contacts, Calendar, Keychain – Pros: good sync, responsive apps, Cons: lock-in.

A few "killer features" that I've been working on:

  (1) A dynamic "daily dashboard" that is a mix of metric graph along with easily manipulated spaces (like sticky notes, diagrams etc).  Currently I use Quip for this. 
  (2) Cyclical relationship reminders (like fitbit exercise reminders, but for your relationships). e.g. "jenny's birthday is in 3 days" or "you haven't called your uncle in 3 weeks, schedule a call".    A ring-based interface would be idea, with colors representing the health of that cycle.

I've thought on this a lot. Some key pillars

  (1) trust  – the most privileged data deserves the highest level of protection
  (1b) dynamic – users can manipulate and personalize as they would their home desktop & office space
  (2) responsiveness – adding a contact or note should take < 100ms . Zero latency – as fast as a pen and paper
  (3) cross-device consistency : a change should instantly sync across devices
  (4) comprehensive search
  (5) polished UX
  (6) mind-map : notes, contacts, events, reminders, photos should all link into each other.

You gave your A and B options poor sync ratings, but I would rate them 'best', because personal filesystem sync now has many reliable options (nextcloud, syncthing, resilio). I've been using those for close to a decade now and I am absolutely satisfied with its reliability compared to Dropbox which I used before. I do not like involving commercial parties in my sync or setting up/using 3rd party-specific sync, which means my workflow which I plan to be using for a significant part of my life my change at a moments notice due to api changes, cost changes and companies going belly up. The great thing with tools that store data in a filesystem is that I don't have to!

i'll have to try nextcloud. My previous "owncloud" solutions were rsync and then qNap's cloud.

Can you talk more about your flow, including what tools you use ? I'm mostly curious about editing text gracefully on mobile devices.

I use Resilio, its a little easier to setup with machines that aren't your own (family, colleagues). I have a number of machines hooked up (a number of shares), and take/read notes on my phone (my personal wiki is in one of the shares, very convenient).

Keeping a node online is the hardest part, I used an old phone previously, now an old laptop as a nearly-always-on-node.

This may be a tangential, but I've recently found that it really is useful to track all/most of these pieces of personal data. I spent a long time trying to figure out the best way to go about this before ultimately just settling on plaintext files and Git. Some of my data goes into CSV files that I generally just manually edit with Vim, some (ie. notes) just goes into literal text files. Sensitive data lives in a separate repo and is encrypted using GNU pass

Since this is how I'm used to keeping track of code and configurations, the friction has been surprisingly low; I'd even go so far as to say I find it fun. I'm sure my specific setup is not what's right for everyone but it feels right for me and I think that's the whole point.

You might want to check out https://logseq.com

I track and note in the same way. This system let's you maintain you data in your private Github repo or local text files (which you can manage using Git, or anything else, yourself).

Note - still kind of work in progress software but worth checking out.

I was looking at logseq earlier this week, but everything I found said you needed GitHub. Do you happen to have a link to the local git method?

I haven't found anything for working locally yet. The landing page says, "Use it locally (no need to login)

It's only for the testing purpose now, please don't write any serious notes without login."

You need to use latest chromium based browsers (e.g. Chrome, Edge, etc) because the other browsers don't support native filesystem api for the record, check this demo: https://twitter.com/logseq/status/1335638230478143494

Thanks for the tip! Looks like this feature is even more stable now. I found this https://www.youtube.com/watch?v=Afmqowr0qEQ

Tangential is nice vocabulary addition to orthogonal in this context. Thanks.

You may find Tiddlywiki [0] useful as it can be synced with github and provides very flexible and easy organization of that kind of information. Or look at Drift [1] for a nice setup

[0] https://tiddlywiki.com/

[1] https://akhater.github.io/drift/

VisiData (https://www.visidata.org) is nice for looking at tabular data on the command line.

That looks nice, but defo not for me, however this reminds me that I used to use the CLI web browser at some point :D

Could you share the structure you've come up with for managing this information? In particular the directory structure, file structure and what data you store in CSV vs regular text files. That would be really great!

I've tried managing a personal CRM in text files, but I struggle on getting a future proof structure. I'll have basic questions like: Should I have one file per relationship/person, or a directory containing files per encounter? An example from someone who has used it for some time would be great to have.

How do you mix git and encryption now?

There may be some new ideas here, but the basic idea behind this is anything but new... things like this have been around as long as personal computers and are often called PIM for "personal information management". I'm not saying this to detract from this particular software but to help categorize it... no need to invent a new acronym.

There is a lot of overlap in this category with the note-taking apps (Notion, Roam, zettelkasten) which have also seen a lot of activity recently... you can think of both of those categories as existing on a two-dimensional continuum of structured vs free-form and statically structured vs dynamically structured data; the OPs app is near the statically structured corner of this space, and "just a bunch of text files" in the dynamic free-form corner. The other two corners may be defined by more dynamically structured note-taking/outlining apps and on the static free-form side we might have things like "private data clouds" (Nextcloud for Open Source).

Whilst in theory this is a great idea to manage such information, it feel like a pretty basic level of management. The functionality is broad but flat, so much so that for my use cases it's not feature filled enough to warrant using. However definitely agree for the right user, this type of application would be great.

It can still serve as base for future adjustments if someone needs such :)

Is it just me or is web-Dev API is just another vector for JavaScript-based malware to expand their territory with?

https://web.dev/file-system-access/

The only amount of life management systems I've ever been able to keep myself using is KeePass. Something about it, and it's syncing model, hits the right level of availability and reliability for me so it's where everything important goes.

Cool idea! Thanks for sharing. Wish it weren't in PHP. Not looking a gift horse in the mouth, nor looking to start yet another PHP-bashing thread, nor criticizing your use of whatever lang or fw suits your personal needs -- I get it, this was a personal project and I appreciate your sharing it. But I have a personal aversion to PHP and for that reason I won't be adopting this for my own purposes. But if I ever build a similar system, I'm glad to have yours to reference for inspiration. Slainte!

OOC if you don't intend on actually contributing to the project why would such an implementation detail impact your decision to use it?

It's not just an implementation detail! The whole idea is to locally run / manage / extend a system for personal data. I don't want to put PHP at the heart of my private infrastructure. Any such system needs to resonate with the tools and technologies I use and/or want to embrace.

Or use docker, and then it’s independent of your host environment. This should be a safe environment for people to showcase their work, and your comment was not constructive.

It’s completely fair to share one’s reasons behind why a project isn’t for them. Docker doesn’t change any of the fundamental concerns with a PHP based project. It might reduce the amount of direct exposure required, but for a piece of software that is meant to become the virtual center of your life, it’s still completely fair to be unconvinced.

I’d argue that Docker just shifts the problem slightly, while introducing a new set of challenges. It’s unclear if the net benefit is even a positive number.

Even if one doesn’t intend to contribute to the core project, it’s going to be necessary to get a bit intimate with the stack over time: upgrades, the inevitable bug that requires some troubleshooting, etc.

I found the parent comment both reasonable and constructive.

You may not have the same concerns, and that’s perfectly ok.

Yeah i know You mean, in my case this is fine as this is the stack that I work with.

That's normal, i know some ppl that use Node as Core, some Java as core, each one takes his own stack i guess.

How does Docker shift the problem and what new challenges does it introduce?

Isn't this more of a proof of concept. I'm pretty sure if it were adopted it would be scrapped and started from scratch anyway.

$ find ./ |grep .php |wc

433 (number of files in project with .php extension)

> Isn't this more of a proof of concept.

Was that a question, or a statement of fact?

The parent simply doesn't need the thing itself if the "written in X" matters.

The commenter has a good heart. Plenty of encouraging sentences. However, the overall comment does come across as an underhanded criticism.

Other than passwords, you could ham-fist most of this into Notion.[0] You can find a lot of landing page/workflow examples on the subreddit [1] and there's a handful of youtube videos on people going through their workflows.

[0] https://www.notion.so/product

[1] https://www.reddit.com/r/Notion/

Pretty much unrelated, but Notion is used in attacks regularly trying to phish my business. Unfortunately I’ve got it blocked on every network.

That off topic point aside, the point of doing this would be to own your data (at least that’s my attraction to setting this up). Why set this up in Notion?

Notion seems more like `work based` scheduler/planner for me.

personal-management-system.pl seems to be down. HN hug of death?

Is working fine for me.

Gah, passwords again!

get a patent on this somehow - keeping your personal data secure in the cloud is going to be one of the fastest and most profitable races we'll see in the next decade.

lol, any sources / articles?

Well if anything patentable is within this repo, you can no longer patent it, since it’s been discovered and sold/given away/licensed

Yeah i don't plan to do this anyway, as this is described in my documentation `about me` etc. But I'm just wondered.

Might be better to link directly to github readme instead, which is fairly informative https://github.com/Volmarg/personal-management-system#readme

P.S. apparently you can just use admin/admin and this will log into a demo session, wasn't clear from the login form!

Hmm cant make it now. Only title change is possible.

Reply on news.ycombinator.com