How Roam Research analyzes product design and team building
thetwentyminutevc.com
thetwentyminutevc.com
I’ve spent years trying different tools to organize information, and none of them feel tailored to my note taking approach. I want something that is light weight, works on every device, syncs immediately, works offline, and functions more like a traditional notebook (focus on creating notes as quickly as possible) while providing just enough tagging and query support to easily find what I need later. Nothing strikes that perfect balance for me, so I started on my own platform.
I plan to launch soon, but if anyone is interested in alpha access and would like to give me feedback prior to launch, I would love to hear from you. Contact info in profile!
What I'm missing the most from it is self-hosting, large file support, full point-in-time history, and programmatic access. I want it to be self-hosted because I don't want to lose my "second brain" just because some company couldn't find a viable business model and went belly up 20 years from now. I want large file support so I can put everything in my life in it: pictures, videos, pdfs, web page archive files, source code etc and I don't want to pay uber-$$$ to store huge files in their cloud. And I'd like programmatic access so I can make my own additions. For example, every link/bookmark I add could download the webpage and cache it so I can search its content and so it's not lost when the website changes owners.
I'm not sure if it meets all your criteria, especially as the meaning of certain features from program to program can vastly differ. However, if I've correctly understood what you're looking for, it matches at least some of them.
Natural hierarchy: notes can be organized into any tree that you like because notes can be cloned (not just copied) into any location.
Very easy to create references with automatic back-references: bi-directional linking is built in and is as simple as typing @notename or Ctrl+L. It also includes automatically generated link-maps and self created reference-maps to see all the links between each document.
Embedded documents: You can embed any document into any other document and have it referenced live and thus updated anywhere it's embedded.
Self-hosting: You can either run it on a local machine or host your own sync server (which also allows you to access it from a browser), super simple to setup.
Large file support: I haven't run into any issues with this, seems reasonably limitless.
Full point-in-time history: From the official wiki: "supports seamless versioning of notes by storing snapshots ("revisions") of notes at regular intervals.". You get to choose this interval.
Programmatic access: Full built-in scripting support.
Every link/bookmark I add could download the webpage and cache it: You can definitely script this yourself but the web-clipper extension for Trilium allows you to save directly from the browser into a new note (text, screenshot, or local copy of site). Not exactly what you're looking for but a derivative that could be modified to suit your needs.
There are so many other features that make it the best note-taking application for me (saved search queries represented & updated as notes in the note tree, full theming support, encrypted notes with timeout function, customizable keyboard shortcuts, active developer that you can speak to, etc).
My multi-year search for an advanced customizable note taking application has finally come to an end. Frankly, I like the application so much that I created an HN account just so I could pass the link on to you.
The best way to understand the benefits of Trilium is to give yourself 10 minutes to read the documentation[2] and take it for a test run.
[1]https://github.com/zadam/trilium/ [2]https://github.com/zadam/trilium/wiki
When I found Roam Research I had two thoughts: 1) wow, this UI is rather polished and the UX is very good too and 2) it would be quite annoying if Roam Research disappeared after I had put in a lot of effort in my database and became familiar with the editor and UI/UX affordances. I know you can export the Roam database to a quite sensible JSON/Markdown format, but if I can't self host the editor I will always be worried about this.
Basically this was a a deal breaker for me so I looked around for alternatives and found org-roam[1]. It's an Emacs package that gives you a similar workflow for creating notes, inserting links to notes, viewing note backlinks and a few other things. It's easier to get into the org-roam ecosystem if you already either use Emacs or vim, but I know a lot of people that have started using Emacs just to use org-roam.
There's a quite high learning curve compared to Roam Research but you have the certainty that it won't disappear from one day to the next :)
I don't want to be too much of an Emacs/org-roam evangelist because it's definitely not for everyone, but here are some of the workflows I do regularly:
1. Have a 'daily notes' file where I initially write most kinds of notes; for example when I change the configuration of a system I'm working on, lecture notes, or pretty much anything else. I can insert inline images and attach any kind of file to each of these headings.
2. Refile those daily notes into dedicated notes and link them to other relevant notes. For example, I would link the file containing my notes for a given lecture to the top-level note for that class, etc.
3. Manage the research notes for my master's thesis. I actually have a pretty cool workflow here:
- I find interesting papers or websites online and capture them into Zotero[2] using the browser extension. This automatically creates a fully populated entry in my .bib file and downloads the relevant PDF or snapshot of the website into my local storage.
- I then use helm-bibtex[3] in combination with org-roam-bibtex[4] to create and easily access my notes on this particular bibliographical source. This note is just like any other org-roam note, so it can contain forward and backward links, etc.
- And finally, I use org-noter[5] to read the PDF and make notes on each page or paragraph. These notes are simply plain text entries on the bibnote. This is pretty huge for me, because it gives me a very focused interface for reading and processing what is being said in the article while I make detailed notes about it (which helps recall and understanding). And also, finding exactly where in the source a given note came from is literally one keyboard shortcut away.
In conclusion, it's a lot of up-front learning and tweaking to get your knowledge management system just right for you, but it may be worth it for some people. BTW, feel free to join the org-roam Slack - there are a lot of people there who are very willing to help.
[1] https://github.com/org-roam/org-roam
[3] https://github.com/tmalsburg/helm-bibtex
What I don't like about Roam is its slow load time. Also writing does not feel snappy enough - if that makes any sense. I feel a slight delay at times between typing a key and seeing the character on screen. Every time that happens I like Roam a little less. I've never experienced that with an app like iaWriter, and as a result, even for all its lack of features, every time I use iaWriter I like it a little more.
I think what Roam is missing for me, besides improving the writing performance, is a weekly email digest somehow summarizing my notes. This could use the bidirectional links and give me a quick overview of what I wrote for the week. That would help improve my memory and be more introspective.
The basic features are taking a default Markdown mode and adding:
1. Follow a link to a relative file at the cursor
2. Make a link to a relative file out of the selected text (e.g., "my big idea" becomes `[my big idea](my-big-idea.md)`
Daily notes is just open a text file with the current date. (This doesn't include back links or transclusion, but I don't think those are make or break features either?)
All in all this is pretty trivial to add to an existing text editor, and if you do, you get all the power of that text editor: E.g., the best search, comparison, and split view systems anywhere. Plus the longevity of plain text.
I can see the appeal of Roam, especially for audiences that aren't already using an extensible text editors. But for people that do already use extensible text editors, I'm surprised there's not more discussion about how easy these features are to add.
I thought this at first as well, but automatic backlinks make a bigger difference than you may guess. It basically turns each document into two things: 1. the document itself, 2. an automatic query of all references to this document that renders automatically as soon as the document is opened. Just like you can have a document with content but isn't referenced anywhere, you can also have a document with no content whose purpose is only to be the place where everything is linked back to, and persistent filtering per document means that you can make it behave exactly how you want. There is no difference between the concept of a saved search and a document, they have been unified.
The biggest value proposition to me is that you maintain each document by itself, only changing things in one place, and other documents/views automatically reflect that. Automatic backlinks with persistent per-document backlink filtering is one way Roam enables this pseudo-DRY thing where you only edit things in one place.
The other big thing it does to enable you to avoid repeating yourself is document embedding: where one line of a document or whole sub-document is referenced and looks like it's embedded in the middle of some other document. Interacting with an embedded document works the same as editing any section (you can still tell it's linked, there are read only modes etc) and the edits actually go through into the original document. Combined with fine-grained sub-document references (e.g. a paragraph or a single todo checkbox) this allows you to create things like temporary daily task lists where when you check off items they actually get marked completed in their related project where you created them, and you don't have to manually synchronize any edits. It's like you get to have your cake and eat it too. For example: do you want to organize your tasks along with the project they're related to, or do you want to organize them in the order you want to do them today? YES.
Not having to repeat myself is by far the best feature for me, and back-references combined with sub-document embedding both enable that to a very large degree. I despise having to maintain the same collection or list in multiple places, or the friction of renaming a document because now you have to fix all the references to it, or even keeping the name of a file in sync with the document header. These are things that I find pure files on disk grates on me enough to abandon the affair.
But there’s a separate point I’m still trying to make: What surprises me is that there’s not a separate, parallel conversation happening about the features that can easily by applied to plain text. E.g., the fact that default Markdown supports simple Wiki links like `[A Link](a-link.md)` appears to be lost on an extraordinary number of users of Markdown. These days I come across someone asking for “Markdown with Wiki support” literally almost daily, now that Roam has popularized the feature.
To put it another way: If people were already managing their notes in Markdown Wikis, and then Roam came along and added some additional valuable features that are more difficult to add to plain text, like back links and transclusion, then that would make sense. But what we have is people lamenting that Markdown doesn’t support Wiki links, when in reality, it already has its own form of them...
I hope they do well
That's where I started. It evolved into a simple CGI program that runs on localhost. That lets me view the markdown files in the browser. When I want to edit/create files, they open in a text editor.
Something that's different is that everything is stored in the tickets database of a Fossil repo (works with the default tickets database of any Fossil repo). That gives me for free (a) a full sqlite database for things like tag querying, and (b) version control and syncing.
they are apparently working on improving the slow load time and also closing down new signups
However, entering notes should also include enough markup to allow quick recall of data at a later date and hopefully allow some intelligence in grouping and analyzing the data (much like your request for digest summaries).
If you haven't already, please send me an e-mail so I can send you the info for the alpha, it's a good balance of what you mentioned!
Offline functionality isn't there yet, but generally my goal was just to build a system that allows you to arrange notes in lots of different ways, especially in ways that make it easier to occasionally share notes with other people (something that I've always found myself wanting to do with some frequency).
Obsidian is mainly about offline Markdown files, being link-centric, and being extensive (plugins). Give us a try if that sounds like what you need!
2) I decided this year my personal notes are too long-term and important to me to keep entrusting to every new cloud startup that pops up.
Right now my notes are mostly in markdown files, synced by Dropbox. More developed thoughts that need to link to each-other I do in Trello (I like their UX and how it helps me think).
I'm looking for a system that has these features, but:
- where I will always be in control of the data (open source?)
- syncs & backups reliably
- seamless on mobile
2) I highly recommend Ulysses for this, which has the best writing experience IMO, can open /save external .md files like a txt editor, and is polished and mature across devices.
Been using all kinds of writing apps / tools since the 90s: AskSam, Netmanage‘s ECCOPro, IdeaMatrix on my Blackberry, Emacs+Org, my various own tools, Evernote, Joplin, Notion and a dozen other tools.
In the end I settled with simple, plain markdown text, some Python + bash scripts, and a nice markdown editor like iA Writer or Ulysses, which is the current editor I am using. Its target audience seem to be writers of all sorts producing large texts. But it is actually a pretty good Zettelkasten tool as well.
Re RoamResearch: I would never entrust my personal data / notes into an online only service (that was just created!) I am old enough to know that products come and go, the most important thing is my personal data.
With the rise of this new breed of knowledge management apps, I have started a repo. to track tools, open-source projects and pros-cons of the Zettlekasten methodology.
It's slightly opinionated and I try to update it regularly. If you'd like to stay updated in this space, you can check it out at https://github.com/krimlabs/z
Another rant: There is no Zettlekasten methodology. This is just a cargo-cult. It's also called Luhmanns Zettelkasten, after a famous scientist and write who was using his notebox with some system. Zettelkasten is just the german word for notebox and there are several well known in history, which all did not use Luhmanns system.
It's probably paranoia, but if I'm journaling my life, business ideas, thoughts on contacts - I just want it to be encrypted on my devices before it goes into the cloud. I want each device to sync and decrypt locally.
And yet Apple Photos does full image library search for your friends’ faces, while not having to have all photos on your device.
Two common ways to do this: (a) bring each piece of content to the client device just long enough to add it to an on-device index, (b) let the clients update and re-sync an encrypted global index as content is added locally.
As Apple puts it:
”One of the best things about Photos is how it protects your privacy. iOS and iPadOS are designed to take advantage of the powerful processor built into every iPhone and iPad. So when you search your photos, for instance, all the face recognition and scene and object detection are done completely on your device.”
With Photos, Apple scans and builds the index locally on device, per method (a). If you enable iCloud Photos, it will also sync the index of fingerprints to labels in (b) style so your names, favorites, and key photos match across devices.
Apple’s quite tickled at making this available to you as a dev:
”On-device intelligence: Apple uses machine learning to enhance your experience — and your privacy — by using on-device processing so other people don’t see your data. We’ve used it for on-device image and scene recognition in Photos, predictive text in keyboards, and more. For example, the A13 Bionic chip and the Neural Engine in iPhone can recognize patterns, make predictions, and learn from experience, similar to the way you do. So your device can create personalized experiences without having to analyze personal information on Apple servers. Developers can use our frameworks, such as Create ML and Core ML, to create powerful new app experiences that don’t require your data to leave your device. That means apps can analyze user sentiment, classify scenes, translate text, recognize handwriting, predict text, tag music, and more without putting your privacy at risk.”
Impossible was too strong. Very, very difficult is more accurate. Tagging faces is much easier than FTS as well, as photos don't change (or you assume they don't change in terms of "who is in this photo"), where as your notes might be changing quite frequently.
Also you're gonna have 1-5 faces per photo max, where as with FTS you're gonna need to store a lot more than that per "note".
>> We blocked this phishing page for your protection:
To solve this, I built https://mmap.it - a short cut driven note taking tool that allows you to save and search without context switching.