Ask HN: How do you manage/organize information and knowledge in your life? | Hacker News Reader

Ask HN: How do you manage/organize information and knowledge in your life?

115 pointsay1n84 comments

This is a general question, but I didn't want to ask specific one because I'm still trying to understand the bigger picture.

We "collect" data all the time, and sometimes it helps us learn something. On one hand we have things that were created by others: funny images, movies, articles on the web, books. On the other we have things creted by us: personal insights, thoughts, snippets of code. How do you manage/organize all of this? Do you have apps/tools for organizing specific type of information and if so, why this particular one? Do you have ideas how to organize/manage this efficiently?

But we're not only collecting information [0], we also learn from it. Most people store all this knowledge in their heads for their whole life, but I think it's not efficient (memory loss, can't search, not always reliable etc.), some people create some kind of personal knowledge bases (e.g. personal dokuwiki/mediawiki on localhost). How do you manage/organize things you learn?

It may seem that I should create 2 different topics, but for me both concepts are connected[1]. What I'm trying to find is an efficient solution to managing (almost) all the information that enters (and already is in) my life. I don't think that there exist a good app for that, I know that there are some that solve parts of this problem (evernote, wikis etc.), but I'm more interested in your ideas on the whole topic, how to approach this problem, where to look, how to think about this etc.

I'm curious about your solutions, ideas and "setups" for this problem(s). If you have any resources (books, research papers etc.) about the topic, I'd love to learn from them. Thank you for your time.

[0] as in bits on the disk, learning can be viewed as collecting new information, I guess

[1] I liked quote from a book, some code from LLVM gave me an insight into a compiler design etc.

View on HN

1. Live simply. As I've gotten older I've found more and more value in not having too many things going on. Gradually I'm sleeping better, eating better, reducing stress, and getting more exercise, all of which is important for the next thing:

2. Memorize as much as I can. It's an exercise; I memorize phone numbers, schedules, people's names, trivia, all kinds of stuff. I've never found anything that matches the flexibility and utility of my own brain. I should use the best tool I have, and that's in my head. Technology is unreliable and constantly changing and difficult to organize and search. I've been practicing this for long enough that now I'm pretty good at it.

3. For everything else, I use a few simple systems: a few sheets of paper to the right side of my desk for scribbling and note-taking (meant to be discarded after a day or two), a pile of to-do to my left, a tab open in my text editor labeled "notepad" for longer-term stuff, and a well-organized directory of documents on my laptop with subfolders like "projects", "writing", "sysadmin", etc. -- I try to keep this directory as small as possible by dedicating time here and there to either finishing or pruning projects.

I disagree that keeping knowledge in your head isn't efficient. I think a lot of people just don't practice it enough. Smartphones and computers and everything else make it really easy to not bother. But, my brain is always with me, doesn't require batteries (well...), can store any type of information I want, and can instantly recall it without having to craft some kind of search query or organize the information in a rigorous way. It is exactly the kind of database storage we all wish we had. It never changes data formats, it never tries to get acquired by a bigger company and then shut down, and it gets reception everywhere I go. If my brain were an electronic tool, I would want to use it all the time. And, the more I use it, the better it works.

(edit: oh yeah, and pinboard. Looove pinboard.)

Having turned 32 this year, I can relate with all of your points.

Make life as simple as possible ("simplicity is the ultimate sophistication"). Use paper when necessary.

Also, try hard not to multi-task. It's been proven the human brain isn't built to handle it.

What system and/or application(s) do you use to help you remember things?

I don't really have a system -- or anything that would make worthwhile reading.

I just practice, a lot. I exercise my brain like a power lifter exercises their body.

For instance, I know the names and some basic information of everyone at the bank branch I frequent. I go up, say, "Hi ____, how're the kids, did you have a good time last weekend?" It's nice for them and a little mental workout for me.

When I'm out and about, I briefly memorize random things: movie showtimes, things I heard on the radio and want to look up, prices of things in the store (so I can price-compare later; e.g. I know Safeway has my cereal for $3.99/box with the club card discount, which is slightly better than regular price but not as good as the occasional 2-for-$4.99 sale at Raley's). I try to really look at the people around me and memorize different facial features and notice different things about them, and if I'm not distracted, then I try to flip through them all mentally when I get in the car. (But this also makes big crowds more overwhelming.)

I play Go. I have a small local club with some friends, and we play religiously once a week, and I play or study more often in my spare time. It's a game where you can improve quite a lot by visualizing the game board several moves in advance, and the game board can get pretty complicated. It's a really good workout for my working memory.

When I'm reading, I try to memorize most of the previous paragraph immediately, memorize the bullet points of the previous page when I get to the next page, and the next day I try to remember the gist of what I read. If I can't, I go back and re-read it until it gels.

Stuff like that. It all sounds silly written out, and like a lot of work, but it's second-nature now. I just do it automatically.

Exercising memory is kind of a two-part process. You have to be able to put things in your brain (typically this just means practicing paying attention), and you have to be able to recall them later. I think the recall part might be the bit that a lot of people forget to do. I quiz myself all the time on what I just memorized, so my recall is pretty good.

Oddly, the harder thing I've been struggling with more recently is forgetting on purpose: there's a bunch of stuff in my brain that I don't need, and learning how to forget that stuff is a trick I haven't figured out yet.

> 2. Memorize as much as I can.

I agree that you should rely on your memory. However, when you die, all that information you have is gone to the world. That's why I started writing things down, for when I'm gone.

Take a look at Roy Underhill, if he hadn't started writing books and opened a school, all of the woodworking knowledge he has acquired would be gone forever (or until someone else dedicated a lifetime to finding it all over again).

Good point! If I ever do anything important, I promise to write about it. :-)

I agree on living simply, everyday I'm trying to do this better.

I also try to use my memory as often as possible and I think it's pretty good (I'm learning mnemonic techniques which are really fun to implement in everyday life). Yes, mind itself is reaaaallly good system, but (at least for me) not all my insights, ideas, notes etc. about particular topic are available on demand, but only some part of them. And if I don't think about one thing very often it can be lost forever. Nevertheless, thank you for taking time to write about your system and giving different perspective.

One of the problems of keeping everything in your head is trust. People forget! Maybe you have a great memory, but eventually it'll forget things. And it should! otherwise there won't be room for any new stuff. I don't think we have infinite storage available; at some point things have to be erased to make room.

But also, keeping that stuff in a system you can trust frees your brain to do other things. Instead of falling asleep reminding yourself to buy milk tomorrow, you can leave it to the system to remember. The system being anything - a notebook, an app, or what have you. There is some good evidence that this kind of delegation allows for more high-level thinking to come about, since your brain is more free to do other things.

Now, as you've touched on - sometimes technology screws you one way or another, and that trust is broken. So if you have no redundancy or backup or plan for that dependency, maybe you'd be better off keeping things in your head.

But you are human and your head is human by extension. it is faulty, imperfect, and not nearly as good at storing things as pretty much anything else. Even if its good now, it won't necessarily always be, and isn't in such good shape for a lot of folk. So I would not paint it in such a rosy light.

I'm curious, Have you done any experimentation with Method of loci?

Yeah, I've messed around with Method of Loci off and on over the years, but it hasn't done much for me. Probably I'd just need to put the effort in to it, and I haven't. My natural tendency has been a grab bag of sorts; if I want to recall something, I think about a category -- "people", "projects", "trivia", etc. -- to kind of prime my brain in some way I don't really understand, and then it just pukes out a bunch of stuff until the thing I'm looking for pops up. Like, if I'm trying to remember an actor's name, it goes, "face -> action movie -> movie box was dark blue at night -> heat -> Val Kilmer". It's usually pretty quick.

You're right though, brains are pretty fallible. If something's really really really important, I do have backups. Usually email, or text, or paper.

...but, honestly? I'm scared to death of old age. It scares the piss out of me. One of the things about it that really gets to me is the idea that I might be 90 one day and not recognize people I care about, or remember anything about myself, or have any idea what's going on around me. The smartest elderly people I know are all very mentally active, and always have been, so even though there isn't good scientific evidence for the prevention of alzheimer's through puzzles and brain teasers and the like, I do it anyway.

Figure out a way to cure cancer and you can be immortal-- just consume as much telomerase as you need.

One special trick I've used is to write an idea down when I have it. Another is to write more often to unearth my ideas. What's more valuable that storing existing knowledge is generating new knowledge.

You won't reach your maximum potential if you only try keeping knowledge in your head.

  The power of the unaided mind is highly overrated.
  Without external aids, deep, sustained reasoning is
  difficult.

  - Don Norman

There is a 2000 character limit in the question box, so here is what I'm using now:

- pinboard for managing bookmarks (database of things that may be useful sometime; probably never) & reading list for articles

- I'm testing tagspaces (http://www.tagspaces.org/) for local files organization (mostly tagging research papers and books; didn't like Mendeley)

- cardav & caldav from owncloud for contacts and events

- anki as a memorization tool (spaced repetition) - from languages to my own mistakes (i.e. "lessons learned", so I won't repeat them)

- for insights, notes, ideas, things I've learned & everything else I use personal wiki (media wiki) on localhost. This is the biggest part of my "system", I have there entries like things to buy someday, current project's notes/resources, useful scripts, configuration snippets, notes from books, journeys, analysis of my own behaviour, personal journal, ideas for startups etc. But it's hard to organize, it becomes a mess very easly after some time. Also, I can't use it on mobile (I don't want to put all this on the web, there is a lot of personal info), it takes time to add new thing/entry (I need to think to which page new piece of information belongs etc.).

- simplified version of gtd as a meta-system managing this system and for projects/things to do

I tried a personal wiki (MediaWiki and SharePoint) on localhost too and encountered the same problems (hard to reorganize, secure mobile access).

So I switched back to plain text, using the Markdown syntax format. Sync works fine with IMAP based email account (note apps on iOS & Android support that too). For visual complex documents I use a WYSIWYG HTML editor and the HTML format. SVN/Git repo to preserve the edit history. And I coded an desktop/enterprise search engine myself to search through PDFs, HTML and various Office formats - similar to the discontinued Google Desktop search and some enterprise search software.

Org-mode is a really good replacement for MediaWiki. You can manage links, tables and lists all with simple and reliable text files with .org suffix in Emacs.

https://www.youtube.com/watch?v=oJTwQvgfgMM

I have totally given up hope on classifying all the interesting things I find on the web each day, so I use an uncategorized save all strategy. I save all opened and downloaded files (mostly PDFs) to ~/Desktop/ and I also bookmark things there (by dragging the URL icon onto the desktop). After a few weeks the desktop becomes a complete mess, so I use a script[1] that puts files into subfolders (by extension), and then I put away this "complete week of research" archive and start from a clean slate (except for one or two active projects dirs).

It's a bit time-intensive to find things, but it's not impossible: let's just say the system is optimized for write efficiency and not read efficiency ;)

[1] https://gist.github.com/ivanistheone/9daa23ae2a7abb472cb2

I like this idea since it solves the issue of easily persisting things you may want to keep around, but wouldn't it limit the types of things you can persist (e.g. can you easily persist ideas/snippets this way)?

Ideas no, but I sometimes drag snippets of text to the desktop too and they get saved as `.textClipping` files.

I think ideas are folder-like structures so I usually create a folder to store code/links/pdfs/clippings specific to that idea, then file away that Idea folder as one piece.

My general advice is to spend less time planning and more time doing, else you might get stuck on what Ze Frank calls "Brain Crack" http://youtu.be/0sHCQWjTrJ8?t=8s

I mostly use Simplenote for ideas and loose notes, and Quip/Evernote for more structured notes. All three sync to all my devices, and while neither are perfect for my needs, they are OK.

The problem comes from keeping everything organized (impossible with Evernote and Simplenote with their lack of structure, impossible with Quip since it's docs/spreadsheets-only) and in a way that works with my mind and workflow.

I'm working on sketching out what a unified personal knowledge management product might look like, which combines a kind of "inbox" of sorts of resources and notes coming in, and also a "personal wiki" with structured docs, and a number of people I know are also thinking about what a 'perfect' knowledge management system would look like. Let me know if you want to bounce around some ideas around.

I've also been thinking about this problem a lot. On my phone now so I can't expand but I would also love to discuss this further with anyone. (Email in profile.)

I'm very interested, will send you an email when I'll finish processing all the information from this thread ;).

What I would like is a way to organize not only written notes but files too, keeping them where they are in my disk or dropbox/google drive without having to duplicate them in another online service

Tagspaces (http://www.tagspaces.org/) is worth checking out if you're interested in files organizaiton

1. A paper notebook. I don't often actually refer back to it, but just the process of sketching things out seems to make them stick. More importantly, once written down, my subconscious starts working on another idea.

2. Poor solution, but I email myself notes a lot. Usually from my phone, then I drop them into a folder in Outlook when I get back to my desk.

3. OneNote, for more long term collections. Microsoft did an amazing job with this product.

I use TextWrangler and several text files along with Chrome bookmarks. I'm minimalist and run a very simple operation so this may not be appropriate for many.

To organize the text files; over time I build an index of categories at the top in CAPS, and each category heading below is also in CAPS. Then when I save / retrieve / cull information I search the category "Case sensitive" to locate it quickly. Once major groupings can be identified and corralled, I separate those into independent text files. It's work.

For Chrome bookmarks I build similar categories but this can get unwieldy if not maintained and subdivided on the regular.

If it matters; all my local files reside in one of two folders (or downstream of them). One is for current "in flux" files & the rest goes in the other "archive" folder. I do encrypted backups on the "in flux" often and the "archive" far less often to external drive(s) & the cloud/online. I have a third "clients" folder but all the files there are temp and go back to their respective servers and I don't backup any of it.

I concur with sp3n concerning over-collecting, often I get back to something and it's already obsolete or maybe not at all. As a result I end up in a data cull session from time to time.

Don't like paper and would love to migrate all to 100 percent online one day.

By the way,is there a solution for windows or linux that would take SQL like queries and allows one to search among files in a computer. Something like :

    SELECT All *.jpg as image FROM /myfolder/** where image.creationDate > yesterday and image.size < 100 and image.filename LIKE TRIP% ;

Then either display the result in a window or as text.

Something that would combine find,grep+pipes into something more "userfriendly".

For CLI: Cygwin `find` with the -mtime and -size options?

For GUI: Windows search can do this with the datemodified:* and size:* operators, or just click the Search tab in the Explorer ribbon to see buttons

You've just described MacOS X's 10 year old Spotlight feature : http://en.wikipedia.org/wiki/Spotlight_%28software%29

I used to use OneNote pretty extensively for nearly everything. Recipes, course notes, thoughts while working on projects, lists of reminders, etc. The main problem with it was that if I wanted to search on something, I'd have to search X number of OneNote notebooks.

I've recently started to use fedwiki instead of OneNote, and things are alrightish. Fedwiki has lots of room for improvement as a wiki, and then as a knowledge store.

I think the ideal personal information/knowledge store would incorporate a tagging filesystem combined with something like OneNote/fedwiki. The tagging filesystem would allow PDFs, movies, etc to show up in searches with fedwiki/OneNote handling the plain text & images. Ideally the client that the user uses is something like fedwiki, where you can have multiple different pages open at once, but also allow you to pull in the PDF/video resources.

For me the biggest downside of OneNote is that I can't use it on Linux/Unix (Wine doesn't always work). I didn't know fedwiki, looks interesting and I'll look into this, thanks for that.

Do you have categories in which single entries are grouped or you just add new thing and search for it later?

Yeah, that was my issue with OneNote as well.

With fedwiki, I have a bunch of 'Table of Contents' pages grouped around themes, Project 1, Project 2, etc, and some pages then link to other pages, like a normal wiki.

Currently fedwiki just does searching based on page titles, I'm (slowly) working on an elastic search backend for it to get full-text search.

I use the web app on Linux - still better than Evernote for basic tables (which is what I find myself creating often).

Three things that have helped me.

1. A little bound paper notebook such as a Moleskine.

2. A mind mapping program -- I use FreePlane -- to store links, including links to files on my hard disk. An advantage is that I don't have to get the organization right on the first, second, or even third try.

3. A lot of the information in my life is not digitized, such as most of the sheet music in the world. So I now rely on my cell phone camera to record a lot of that stuff.

Amusingly, when I was in grad school, it was still considered to be an open question whether a person should get their own computer. The university computer store had a little guide, and the most memorable advice -- which certainly rings true in my life -- was: "Don't expect a computer to make you organized. If you have a messy desk, you will have a messy computer."

What I do is:

- For meetings, events, social obligations, I carry a small paper-based diary.

- For ideas and thoughts, I write them down. The idea is that the act of writing something down aids recall. For example, story and blog ideas go in a A5 notebook that I carry around with me, and I commit them to a digitally backed up document as soon as possible. Note-to-self lifestyle advice goes in my bedside drawer, to check on if I ever feel like I've forgotten anything.

- For things to learn and interesting articles, I don't do anything and keep my fingers crossed that the salient bits will have rubbed off on me, lurking in my subconscious and subtly improving my life forever after.

I'm not 100% sure that last one works quite that well.

This is pretty close to what I do, but I also try and keep track of what I read. To do this, I just bookmark everything I read in pinboard so if I remember some tendril of a thing, I can search for it. In general my memory is horrible, every little bit helps.

What would you do to improve how your subconscious works?

I use Evernote. Because it can handle multiple types of content. The most useful information I keep in there are tutorials. Mostly written by me. How to do things that I only have to do a few times a year. Basically recipes for anything, not just food. The stuff I do repeatedly I just remember or automate.

Passwords are a good example of this. I can deduce them(based on a formula), but I don't want to do that every day, so I use 1Password.

But it's not bad to just forget. There are quotes, links, funny pics in my Evernote that I've never used. So I'm now much more selective in my note taking. It's less stressful.

To organize myself I use Secretweapon [1] (GTD for evernote) and Folder-System [2] (an hierarchical folder system to organise personal documents based on GTD, making it easy to predict where something is stored). The initial investment (file renaming) pays off very fast.

[1] http://www.thesecretweapon.org/media/Manifesto/The-Secret-We...

[2] https://github.com/we-build-dreams/folder-system

org-mode in emacs

Simple text file that's actually more functional then almost any other solution, designed around allowing you to create and adapt new workflows and WILL be available and useful to you for decades.

https://en.wikipedia.org/wiki/Org-mode

I approached this problem and decided to try and solve it for myself just over a year ago. Firstly, I decided that instead of knowledge being segregated by medium (notes -> evernote, bookmarks -> Chrome, etc), it should be organised by purpose.

The most important purpose, I decided, was personal goals. Knowledge/information which is relevant to helping me achieve my own goals is the most important thing I should be focusing on, and should be extremely well organised and easily accessible. Any other interesting info that falls outside that is a bit of a shame to lose, but ultimately just a distraction and clutter. For this purpose, I developed this tool: https://nachapp.com

I believe the next level of information down would be general learning/knowledge. Stuff that doesn't fall under any specific goals, but is still useful information to know and understand (and may in disparate ways tie into core goals). For this, I'm currently using https://pinboard.in, although it's not ideal as it's again limited to a single medium. I have a solution in mind, but haven't started developing yet. If you're interested, feel free to get in touch and I can keep you updated (contact info in profile).

I've come to find that everyone has a different solution, and lots of people have different priorities. A client of mine keeps everything in text files. I know another that does everything in excel. Others use some combination of apps and services (dropbox for files, omnifocus for todos, evernote for notes and articles) and that works ok for them too.

After trying several solutions, I ended up building a SaaS to manage everything. I found most things out there are fairly boring, and are not at all as powerful as what I wanted. It's basically a brain for my brain; so i can remain a scatterbrain and it can tell me when its time to water the plants, or if food in my fridge is about to expire. But it also handles all my notes, important files, time-series data, and historic dates.

But really, you have to figure out whats important to you and what system best aligns with that; and in the end you'll likely need to make a few tradeoffs to get something working.

But I think there are definitely some principles you can apply to any system you use; I can't recommend Getting Things Done by David Allen enough. His methodology is great, but even if you don't like it or can't use it for whatever reason, there are oodles of great tips; and it'll make you into a natural project manager / information guru.

Do you have it online? care to share?

I write a lot and have a lot of text files scattered all over my desktop with ideas, plans and notes. Everytime I start a programming project I also end up with a lot of text files with requirements, design decisions and implementation details. I kept all these ideas in plain text files because, none of the existing tools really fit what I wanted. I ended up creating a program to help with this based on the ideas of Doug Engelbart.[1] The program is still experimental and just deals with text at the moment but I still find it useful.

In terms of research papers, you may want to look at the ideas of Doug Engelbart. The process which you speak of, of collecting information and learning from it, Engelbart termed the CODIAK process. There is a section describing what CODIAK is in this paper[2]. (click on the CODIAK Process link in the table of contents). Engelbart speaks of this process in terms of groups and organizations, but the ideas apply to individuals as well. Engelbart's goal was to create an integrated "knowledge workshop", where all the different programs for organization everything would be integrated together and act as an extension of the human mind to augment people's abilities to collect and digest an ever increasing amount of information and knowledge. There is a lot of work left to be done in this area, but it is an important problem to solve.

[1] https://github.com/smarks159/hyperdocument-system-wiki

[2] http://www.dougengelbart.org/pubs/augment-132811.html#6

After trying a lot of software I returned at the end always to Evernote. I always missed good rich text and image support and easy web page saving in all other products. Evernote has a great web clipper (also works for Android with Everclip) which saves sooo much time and great IFTTT integration for interaction with other services. Especially having everything synced and editable on your phone is something the other solutions are missing.

I've been using Kifi[0]. It lets you collect any webpage thats interesting, tag it with multiple tags, and search your whole collection based on tags/content of the page. I highly recommend it.

For example, I want to learn more and more about distributed systems, so when I see an interesting article, i tag it. When i have some time, i go through the relevant tags.

[0] https://www.kifi.com/

This is something I am constantly trying to get better at.

I have recently started using jrnl: http://jrnl.sh

I really like it because you can export it any many formats which I think would come in handy later to import the data, say a database. I always like the idea of a flat file for right now so you can ack/grep on the command line.

The cons are not great for assets such as images.

Looking forward to reading some other solutions.

Depends on the level of activity/energy required:

Things with well defined start/end dates or with very high priority/high cost of missing (ie flights, parties, reservations, deadlines): calendar (I use my iphone calendar with alarms if necessary)

Things with less well defined start/end dates: starred emails/browser bookmarks toolbar

Menial things that need to be in the near future (ie groceries, laundry, shopping): Leave something out of place as a reminder, or put in calendar with alarm if urgent

Things with low priority/low cost (ie things to read/learn/listen/watch, fitness goals/accomplishments, future trip plans, long term plans, misc notes): iphone notes

Also keep a notepad at everywhere I work regularly (home desk/office desk) and in my backpack with notes

I also feel its very important to maintain 0 unread emails in all of my inboxes. Makes it much more feasible to stay up to date on everything and avoid missing anything important. 90% of the time I'm awake, I will read any incoming mail within 15 minutes and respond immediately if necessary.

Org-mode with mobile app, and a grimoire file. It holds all snippets, captured thoughts, bookmarks, to-read, etc...

It's also responsible for my spaced repetition system, my work agenda, my personal agenda, blog post ideas, project ideas, grocery lists, todo lists, etc...

It's extremely powerful and my life is forever changed because of that software.

All appontments meetings personal and work - in a single Outlook calendar. All work related notes in a single OneNote workbook (text searchable) All code files/projects - on Onedrive in cloned repos from Github, so I can include them locally and commit to GitHub as needed. Lastpass for passwords and credit card info.

One unique thing I started doing, I use this program called ClipX which lets you have a popup of your recent clipboard entries. Its will let you save to a text file. So, everything I clip, code snips, passwords, etc.. goes into a single text file - so I can just Ctrl-C something and forget about it. Six months later I just do a text search in this file. Its gets big (10MB or so) and I back that file up and start with a new file every year.

I don't have a lot of information that I need to handle, but I use zim (http://zim-wiki.org/) as a desktop wiki. It's working well for my little needs.

There is also camlistore (http://camlistore.org/) which is trying to be the recpipient for every content one wishes to store: images, music, PDF files, bookmarks, RSS feed items, ... It's working well in the "aggregating" phase, but still a little bit lacking in the "organizing" phase (which may very well be the work of third-party application). For the moment content is accessible through a simple search engine though.

I have a bunch of text files in folders on google drive. The cool thing about google drive is you can sync it and it propogates to any web enable device.

Keep your life to three to five major things and this works really well.

Also, Don't go more than two or three folders deep. If you do you're organizing wrong.

I also scan or photo all of the paperwork I get throughout life put it in Google drive and throw the original away.

Also, you can put all your photos up on google drive. (Just not very private photos..)

Note: security/privacy concerns if you do this with personal stuff like medical or financial docs.

There's ways to encrypt it all on google drive just don't really have enough important info there to care.

Still figuring this out - I have DevonThink to collect/catalog but I find that I don't often get in there to search, so that information basically rots. Same with my web bookmarks, many are several years old. As time goes on I find that information is basically useless unless I either internalize or act on it. So I tend to use information to adjust either my anki decks or my goal maps, and then I try to deliberately not save the rest... although that is still difficult since some of it is still just so tempting to flag by saying "gosh I will really want to refer this later after I do x, y, and z"...

I never used DevonThink, can you tell me what are the your personal biggest pros and cons of this program? Also, what do you think is the biggest problem of this/similar systems?

I can sympathize with bookmarks problems, not only they rot, but after few years there is a lot of 404 (pinboard provides archiving bookmarks but that is a paid service).

Do you have a lot of Anki decks of just a single "random things from the bookmarks"? Do you use Anki every day or rather infrequently?

I use DevonThink as mostly just a clipping/storage app, but it has tons of utility beyond that - seems a lot of grad students use it for managing resources they later write papers from. I chose it over Evernote because it does more. The main downside of it is that it doesn't yet have a good mobile syncing solution, even though they've been working on it for years now. But it has a ton of UI customizability for searching and browsing, and search works well too.

The main problems with all these systems are just that we use them to attempt to augment our own intelligence/utility based off of some concept of who we are aiming to be in the future, and then as things change for us personally, our aborted plans lead to a lot of rot that doesn't clear itself away automatically. So only some small percentage of what we collect ends up relevant.

I have a lot of individual Anki decks, which isn't recommended. But I create a filter deck that rebuilds from the other decks every day and properly randomizes the cards, and only study from that. I do it every day except when I forget and then I catch up later. My cards are mostly along the lines of music, logic, math, stats, programming, and rational thinking.

Ironically, one of my unfinished personal projects is a project manager, so I can organize myself better, but I find just pasting random stuff into a notepad++ tab does the job most of the time. I also have a ton of browser bookmarks organized by nothing in particular (most of my logins actually say "Login") and a lot of random stuff in my downloads folder, repos I found interesting, pdfs, etc.

So I guess my answer is "poorly," but somehow it seems to work.

I have a notebook (moleskine atm, but I've used a lot of other brands/kinds) for sketching out and expressing ideas/insights and I use Evernote for long-term storage, project management, and to store reference material. I also use Safari's reading list to hold on to things I want to read, but don't know if I want to save permanently.

Edit: I seriously keep everything in Evernote, whats up with the down votes?

Don't know why somebody downvoted you, I've upvoted your post. I also use notebooks, mostly when I'm away from computer but as you, I'm using it for sketching and brainstorming sessions.

Do you regularly review things in your Safari's reading list or it keeps growing? Because that's the problem in my case.

My reading list is pretty big. I keep articles there until I get around to reading them, if they seem like something I want to keep around I clip them to Evernote and remove them from the list. In some cases I'll just move them into bookmarks which are mostly unsorted, as I use them primarily as a durable form of browsing history.

I use google docs. I have a bunch of google documents i regularly use for work

- one where I keep track of my work: it's half a todo list with bullet points, half notes about my progress for meetings - one where i write ideas, things I want to dig into later

I write in them from bottom to top (so that my last entry is always on top). I basically use them as paper notebooks.

Actually if it wasn't for URL copy pasting, I think i'd use paper and pen.

I just dump it somewhere (OneTab extension for web pages, random folders for stuff, note-taking for anything else).

Most of it I'll never need again (and probably look online for it first anyway, if ever), so it's really just to satisfy a compulsive need to have the impression I'm not "losing" anything. Rarely did I ever dearly miss anyhing on accidental deletion of my "stores".

Nothing beat a filesystem to organize stuff, it also happen to be great way to find stuff fast..

- Folders like \doc \tutorial \ideas \recipics - Store everything in .txt and .odt - Replicate to favorit cloud storage

- For comunication: Fastmail, Skype

- Contacts & calender: Fastmail

- Keep a diary in a mail folder, I send myself a mail every now and then with a subject to an alias, that gets sorted into a "diary" folder

Here is one idea.

http://lifehacker.com/im-ryan-holiday-and-this-is-how-i-work...

The take-away there is index cards. I like the idea of information being broken down small enough to fit on an index card. Reading a book, it could be one point, one quote or something of similar size.

Will read the article, thank you.

In the past I've tried to use POIC system[0][1] but it wasn't really that efficient and the biggest 2 problems were search (mostly full text, I could find entry from a particular date quite easly) and portability - over time it becomes a big physical thing you need to store (I move quite often).

I agree with you about dividing (almost) everything into single pieces of information and adding to the system, but I'm still trying to figure out how to do this well.

[0] http://pileofindexcards.org/wiki/index.php?title=Main_Page

[1] http://lifehacker.com/the-pile-of-index-cards-system-efficie...

Not an expert but here is what I do:

1. Dropbox for all the read-only ( Images, PDFs, Books, important papers).

2. I use IFTTT to autobackup a specific folder (very imp one) from Dropbox to Box.

3. Something I need to edit, like spreadship or docs. GoogleDrive works very well.

4. I keep most important files on atleast two services.

5. IFTTT works great to collect data automaticaly.

Edit: Goodlooking line breaks inserted.

Thanks for sharing. There are a lot of cool ideas here. I've had the most sucess with a custom notebook approach using QR codes to stitch together analog and digital information:

http://www.drbunsen.org/custom-notebooks/

I use vimwiki to collect, store and organise snippets links and references, then sync it to all my machines with unison. Helps if you're a vim user!

https://github.com/vimwiki/vimwiki

For thoughts and ideas I use Evernote and it works very well. Have over 3000 notes now and it's fun looking back on thoughts, feelings and challenges I had years ago and how they've changed and evolved since then.

Previous discussion : "Designing a Personal Knowledgebase" - https://news.ycombinator.com/item?id=8270759

I wrote in this thread about using a Wiki I wrote: https://news.ycombinator.com/item?id=8753599

Just started reading. Most of the items on your list of things to fix I was thinking about for quite some time. Besides the comments, maybe you could write a blog post about your system and experience with it? I for sure would be interested.

Do you think that solution to the whole personal knowledgebase problem should be solved from a single app perspective or is it better to think about this as a sort of ecosystem - one "module" for adding new things to the system, other for statistics[0], sync etc.? Also, did you consider semantic part of such system? So we can write programs that can make sense of all the things we have in the system?

[0] I was actually thinking about using machine learning to somehow finding the most important things in the wiki and suggesting tags/place in hierarchy of new entries

Actually my main takeaway is that organizing information shouldn't occur in a single app. It needs to take advantage of the web as much as possible. The model I want is basically small "apps" joined by hyperlinks -- exactly what the web is.

The primary app is a wiki with unstructured text, outlined/indented text and hyperlinks. But, as mentioned, information comes other forms too: structured data like bookmarks, spreadsheets, images, etc. The web lets you express all those things (spreadsheets being the weak link, since it involves executing code). Reinventing it all within a single app would be a lifetime of work.

The web already has common patterns and facilities for log analysis and so forth. If you just make it a web site, you can track all your actions with web log analysis. Whereas if you make it some "app", then you have to reinvent the analytics.

I definitely considered doing more textual analysis, stuff like PageRank, and even semantic analysis. I think it's interesting to explore but I don't the cost/benefit works out right now.

I have over 2000 pages, but probably 500 or less are "active". That can mostly be managed with full text search, and nothing smarter. You just have to make the search fast, which isn't hard with that amount of data.

But, if you want to write more code and programs against the data, a web site is a good format for doing so. I basically use the Unix philosophy of independent programs and data, rather than the "app" philosophy of data locked up behind code / user interface.

My philosophy is just to optimize for speed and ease of both writing and reading. And then you use it more. And the more you use it, the more valuable becomes. Just like with learning itself, you start to make more connections once you have some existing knowledge/content to graft on to it.

I think the most important thing is the content, and not the tool, and as long as the tool is lightweight and fast, you will keep up the habit of entering content and (just as importantly) retrieving it.

That link is full of wisdom, thank you for posting it! I never met anyone who wrote their own wiki and used it for so long.

I was surprised the biggest thing for you was speed and full-text search, and also by this comment: "I think people forgot how to make a simple web app with a form and plain buttons."

The biggest thing for me is learning. Not so much categorizing everything correctly.

A combination of ( sorted from more to less relevance):

Trello

Workflowy

StackOverflow's favourites

Pocket

Goodreads

Delicious ( I hardly check it)

Some self promotion, I'm tackling this problem with my side project: www.faqt.co

badly, i have hundreds of bookmarks, articles, documents that i haven't looked at since adding them - not quite sure what i'm waiting for but i spend a lot more time collecting them then actually reading them

Yes, I think this is a common problem. We collect things, because they seem worth reading/watching but never return to them. One thing is that some effort is required to read/watch those things but other thing is that we forget that we even saw that particular article/video. So, maybe bookmarks should have a reminder mechanism built into them of some sort? Sombining Spaced Repetition with bookmarking seems interesting.

This is further fuelled by recalling something (e.g. an article you read), and not being able to find it again.

For example, a while ago I read an article that talked about peanut allergies being less prevalent/severe in Europe than the US, and Europe having more cases of other allergies (like apple allergies). It then went on to talk about different pollens that may react with our bodies to trigger certain allergies. I can't recall how long ago I came across this, or where (could have been HN, Slashdot, Digg, etc). I've also not been able to put together a search to turn up this article in Google. Had I been better about bookmarking, maybe I would be able to find this article even if it took a brute-force search through all of my bookmarks.

Yes, I know the feeling. So even if I know that I won't return to 90% of my bookmarks, I'm adding all of the interesting things so if I'll need it in the future I'll have pretty small set of links (rather that billions from google).

Another thing is archiving all those bookmarks, since after few years a number of them is 404.

If I find something remotely worthy, I bookmark it with the nearest button available. Most browsers have one near the address bar somewhere.

Trello - ESSENTIAL kanban system for project management and daily organization. I'm an absolute fanatic for Trello.

OneNote - The greatest program ever made. OneNote isn't useful for short-term to-dos, but I couldn't live without its organization of my long-term work.

Pocket - I never have time to read articles, and they're unpleasant to read on a squat laptop screen in any case. Pocket lets me read them on my tablet, in the subway, with gorgeous formatting.

Google Calendar - The whole Google ecosystem is ridiculously useful for organization, including auto-additions to your calendar from Gmail and Google Now's intelligent suggestions.

Gmail - Particularly the five-tab filtering. Holy shit. I had no idea how disorganized my inbox was before I could filter away the dreck.

Google Keep - It's been displaced somewhat by Trello (in terms of grocery lists, etc.) but it's still great for medium-length notes that you'd like to read on the subway or something. I store a huge amount of poetry in mine.

Google+ - The BEST place for photos. I don't understand the fashionable hatred. G+ is by far the best photo backup/organizer/enhancer/sharer I've ever seen, and the G+ social network is WAY better than Facebook's clunky organization. I'm amazed that Facebook is so bad with photos... isn't that pretty essential for social networks? G+ is just too good to be ignored forever.

GDrive - Corollary to G+. Putting photos on GDrive automatically uploads them to G+, which again is just excellent. Google DEFINITELY needs to upgrade their 1TB limit to "unlimited," though... and offer auto-deduplication for identical photos for the billions of us with redundant photo hoards. Right now, GDrive prices are the highest on the market for limited storage and no extra features like deduplication or auto-organization. I still pay for GDrive just for G+ photo features, but Dropbox offers equal/better functionality and OneDrive offers much better value.

Kindle - You never realize what a burden paper books are until you have an alternative. Ebooks are incredible.

Calibre - Especially if you have a ton of academic papers or studies to pore through, organizing them in Calibre makes life infinitely simpler.

Spotify - Outsourcing my music collection to streaming services is GREAT. I'm now trying out Google Music, which lets me upload 20,000 MP3s to supplement the holes in Google's collection. (Perfect for unpublished songs and so on.)

Steam - The original Spotify for video games. All the same benefits, plus amazing sale prices that have forced me to buy way too large a collection.

edX - Coursera and Udacity also, but any of these are gold mines for organizing self-paced education.

Pidgin - Still the best instant messenger, though the shift toward closed networks is making it harder to use. That Google still allows Pidgin to access Hangouts via (limited) XMPP has kept me faithful. I don't understand why anyone would use something like WhatsApp and its crappy clients.

- - - - -

If anyone from Google is reading this, particularly from the Tasks team... just buy Trello already. GTasks is woefully inadequate, particularly when compared against Trello's unbelievably powerful kanban organization.

Also, I'm a little disturbed by how heavily I depend on Google. I don't see any viable competitors, though -- Outlook lacks Gmail's tabs and themes, OneDrive lacks GDrive's functionality, Facebook lacks G+'s insane photo features, WhatsApp lacks open APIs. I hope Microsoft steps up soon, because I don't see any other real competitors against Google dominance of web services.

I second this, Trello totally decreased my overinformation-induced uneasiness, and happened to help me summarize information. Of course not all kinds of information fit there. I found out it's great particularly for information I want to be in contact with quite often.

Also Trello Inc's public development board(https://trello.com/b/nC8QJJoZ/trello-development) is just genius, imo.

I recently started keeping track of a lot of stuff in Tiddlywiki (in server mode). I keep a browser tab open to localhost:8080.

I've used Tiddlywiki in the past but I didn't quite like it (just my experience, I know that a lot of people use it). Switched to dokuwiki and after that to media wiki and I'm using this now, but as you can guess from the question, I'm trying to find/build something better for my needs ;).

How do you use Tiddlywiki - do you often search or do you know where did you put some note? What in your opinion are pros and cons of this wiki? What do you think is the biggest problem?

I just search. Between text and tags, I always seem to be able to find whatever I'm looking for.

Pros: It's easy to set up, and has been robust. I like it that it stores its data in the filesystem, in a location of my choosing.

Cons: It seems to use a somewhat unconventional formatting syntax. I find a lot of the terminology pretty opaque. Having had a few scary configuration-related incidents, I don't try to push it to do exactly what I want anymore.

I'm glad to have found Tiddlywiki and set it up, because at the time, I was storing notes in a thousands-of-lines-long text file, which just wasn't working. But I should probably resurvey the scene and see if there's anything better.

I recently started trying out both Tiddlywiki and Zim Wiki, and I'm starting to like Zim better. With Tiddly I didn't feel like I had stuff really organized. Of course you can search, but it didn't feel right for me.

With Zim everything is in plain text files, and it integrates with version control. This makes things more transparent (to me at least), and you can still use some useful things like filtering with tags, searching, todo-list overviews, etc.

Reply on news.ycombinator.com