Show HN: Open-Source Memex – Alternative Approach to Roam/Obsidian

This blog is a summary of a fun 1 month adventure I had with Knowledge Management Systems and building a POC that I thought had potential. It was inspired by so many of the products I see on Hacker News.

I’ve open sourced all the code + written down some notes on my insights/ architecture. Just a warning, the code leaves much to be desired as this was a mini project over 2-3 weekends.

At the moment, I have 3 key layers of the application.

The first is Chrome Extension which allows data collection. It tracks all sorts of data such as attention, time on page, scroll/ click and hover behaviour. It then sends this data to the Electron app.

The Electron app receives this data through an API and saves it down locally using PouchDB (data structure is compatible with Apache CouchBD and thus allows easy cloud storage).

Within the Electron app, I use an Express/ NodeJS web server to expose endpoints for functionality with the front end/ receiving data from Chrome Extension.

For the front end I use ReactJS.

If you’re interested, please read the full article! There are lots of videos of the application in action.

This is a great concept, especially the link tracking.

One of my early research papers was about a tab reorganisation UI that tracked the links you clicked to reorganise your tabs to follow your train of thought. In most cases, the flat organisation is the worst, whereas if you follow my pattern of clicking links and moving to different tabs over time, you're halfway to describing (in a way that I can pick back up) my stream of consciousness.

Didn't get too far, for something like this to work it would have to be well integrated with the browser - and with new privacy restrictions, you'd have to end up recompiling a new browser to actually provide enough functionality.

Have you thought about ease of resuming where you left off? Biggest problem for me with Memexes isn't administration (even though it is huge and exponential as you say), it's that the longer it's been since I documented something, the harder it is for me to get back into the same mindset as I did then, with all the pieces still intact and connected.

Seems to me that finding a good representation of the internal mental model will help get over this.

Could I add further constraint. These are really interesting ideas, I for one would be prepared to contribute to a project developing them.

Given the amount of investment in these systems, they need to sufficiently future proofed as to be useful 10 to 20 years from now.

> Insight 3: Knowledge discoverability is a problem. Say your KMS is 5-10 years old with thousands of notes/ pieces of information, how do you sort through it?

I think this is one of the biggest problems in the space right now. We have smart inputs but not smart outputs.

Best case scenario, I would like to recommended knowledge based on the context of what I was working on or trying to achieve at any given moment.

I'm really glad you put a lot of focus into automatically tracking the things the user is likely interested in. I tried using Obsidian, but it felt like I was spending more effort remembering to save information and create proper back-links than I was actually retaining any information.

I've recently started working on something similar as an excuse to learn machine learning, but it's still mostly vaporware outside the firefox extension I wrote. I think that by saving some basic metadata (when a page was viewed, what browser was used to view it), and using ML to judge how similar the contents of a page is to another, it should be able to automatically create links between related information. Ideally, it'd be able to handle information outside the browser. For example, if a log file is saved, then a web page is viewed with similar contents to the log file, it would be able to detect that the web page is probably a reference for the log file.

Like I said, it's mostly vaporware, but I think that products like these are going to be the future of collaboration tools.

This is unreadable for me, and I'd welcome any suggestions as to what I can do to my setup to make it readable by default. To see why, here's a section of screenshot:

https://www.solipsys.co.uk/images/AnotherUnreadableSite.png

Unreadably light grey on white.

All suggestions welcome ... TIA.

This is really fantastic. I've played around with Obsidian periodically, but it's difficult to groom and maintain. This approach seems more manageable.

I'd like to start using this, but at the same time you likely won't be maintaining it.

Final point: we really need a memex/knowledge graph ecosystem with easily interoperable components (e.g. browser extensions, book highlighting, etc that can all feed into one of many viewers).

I was worried about yet another note taking app post on HN but I have to admit the Chrome Extension/Inbox part is a fresh idea and really intriguing.

There is also https://getmemex.com/ which runs as a chrome extension.

My old personal KMS had a feature to load the local chrome history and display my activity on the daily timeline. I was mostly interested in understanding my day, recalling and to revisit.

Great experiment you have here, thanks for sharing!

Have you talked to Andrew Louis about his Memex?

https://hyfen.net/memex/

Definitely someone you should reach out to in this space if you're interested in seeing other approaches.

This is interesting. We at OrgPad.com try to do something a bit different but still in the related/ almost same space - we want to make a system usable for normal people, not for mostly highly educated and concentrated hackers/ IT professionals. We don't do any activity tracking, you have to be deliberate, what to write down and what not. We also currently have no way to evaluate information in the units e.g. for some kind of hierarchy or automatic sorting. These features will eventually come, but we are not there yet. As I said, our users are mostly people with expertise in other fields than IT, like teachers, students or managers.

OrgPad.com is a SaaS tool that you can use for free. It tries to do away with as much hassle/ non-sense as possible by really focusing on information and relationships between those. You can connect units as you want, you can put into units what you want. The layout is topologically stable, but the absolute position can change slightly e.g. if you open up a unit which has a large picture or something. This is a completely new algorithm that understands a units area and therefore knows, if a link is crossing it or not. A killer feature is being able to do "a path through the graph" which is basically a presentation/ slideshow of sort. The nice thing is, you don't have to transform your knowledge to a different format to be able to present it. We have found that normal users are a lot quicker creating such a presentation compared to e.g. Microsoft PowerPoint. Oh, and of course you can easily collaborate on one OrgPage with multiple people e.g. by sending them a link for editing or adding them to your team. If you want to just have a look and not create a login: have a look at some of the public OrgPages https://orgpad.com/list

Currently, we are writing our own editor, that should be much simpler than the current one and therefore integrate much better with the whole concept. When it is done, editing on mobile will also be possible. Mobile is currently read-only, but you can at least upload photos/ videos and sort them later, when you are at a computer which is a big help for us and our users. The whole thing is developed in Clojure/ClojureScript so the idea of simplicity really was an inspiration.

I believe knowledge discoverability can only be solved with NLP. Parse every article a user reads, extract topics, keywords, key concepts and store it is whatever your storage is, in searchable format.

OOT, in Indonesia, if you replace x with k, it's a slang for female genitalia. A very strong word you don't want to say out loud at work.

The 'technologies used' section should've mentioned 'antd'

https://ant.design

Very cool! Hope you might consider adding a Firefox extension too

Very cool, even if I hate electron. :P