Serendipity is too important to be left to chance (1996) | Hacker News Reader

Serendipity is too important to be left to chance (1996)

web.media.mit.edu

131 pointsthunderbong35 comments

I'm working on a system similar to this to satisfy my own needs but can open up to brave alpha users if requested.

I've a concept of boards/themes where you can add structured information like people (to remember contacts in context), bookmarks, thoughts, tasks and even RSS feeds to stay up-to-date on a theme (e.g battery tech). Emails and calendar will be coming soon so that your personal corpus is voluminous and of high enough quality to make a personal GPT actually useful. E.g "who was the CTO I met at the dinner last week".

FWIW I'm already a founder of a company so this isn't intended as a commercial pitch.

Years ago, I submitted an admittedly half-baked version of this to YC. The concept is interesting, but the implementation options are:

a) require considerable and routine manual data entry from the end-user

b) suck up every bit of data from e.g. email, phone, messaging apps, banks and end up with some privacy/security disaster

Anyway, there are all sorts of things you could do with that data, like trawling for novel links in your social graph.

Use b) with a local datastore and attached LLM and it sounds like a great solution to me. I would 100% give this a try if someone already has something rigged up on GitHub or try building it myself if not. Should be easy to run on a modern machine and I hear open source LLMs are constantly improving.

I'm leaning into this without a commercial consideration so I don't need to overthink it, it's just something I want for myself.

I agree point b is a minefield and given my info is on the system, I've got skin in the game to make it secure.

I'm doing something like this with https://histre.com/ I'd love to try your tool, and it would also be fun to chat about this if you like (k@histre.com)

I've been doing something similar based off of what I've been calling "intentionally overspaced repetition". I'm interested as well in how you're going about it.

I'd love to talk more with people who have done work in this space. Drop me an email at "jb at kulor dot com" and I'll explain more.

Hi, I really need something like this. I work on simultaneous engagements (mgmt. consulting) and travel substantially. This would help me organize better. Please let me know if I can try it.

Hi, drop me an email at "jb at kulor dot com" and I'll share details. Not ready to publicise the project.

Yes! Would be interested in trying it out if you are looking for alpha users.

Have you tried Confluence ?

Yep and Notion, Obsidian, Foam etc. All fine tools in their own right.

I’d be interested in something like that, where do I sign up?

Hi, drop me an email at "jb at kulor dot com" and I'll share details.

I am sure some of you here will enjoy the etymology of the word serendipity.

Apparently the word was inspired by the fairy tale "The Three Princes of Serendip" [1], the latter being the Persian name of Sri Lanka. In the story, the princes would constantly find interesting things by accident.

[1] https://en.m.wikipedia.org/wiki/The_Three_Princes_of_Serendi...

Knowledge management reminds me of The Codeless Code on documentation: http://thecodelesscode.com/case/215

PKM [1] concepts seem to have taken root in the mid 90s at places like UCLA and MIT.

I think in a way social media is the PKM backdoor. People share near-everything about their personal lives in exchange for likes. On the flipside, you have a pretty good record of what you've done, where you've been and your feelings at the time.

I think LLMs will give new breath to PKM.

[1] https://en.wikipedia.org/wiki/Personal_knowledge_management

The modern equivalent of this is basically Obsidian and the "second brain" concept

Not like chatgpt, which is a random garbage spewing agent leaving it to chance

Or Org-Roam, to stick with the Emacs connection.

Emacs is very awkward to use in a mobile phone sadly.

It's not as seamless as I would like, but I manage pretty well with Emacs in Termux and files synced via Syncthing.

Orgzly https://orgzly.com/ a nice way to do org mode editing on the go. Gives you notifications for reminders/deadlines and you can press on tick boxes to toggle them etc. I use it with syncthing.

Typing anything of length and substance on mobile is very awkward, to be fair.

Now, with the voice to speech button, it’s quite easy to get length. Substance is still tricky though.

> The current Remembrance Agent uses the Savant information retrieval system developed in-house by the Jan Nelson and Bradley Rhodes. The Remembrance Agent runs through emacs, a popular text editor. The user interface is programmed in elisp, and the results are presented as a three line buffer at the bottom of the window.

There is a nice retro screenshot of Emacs after this. Looks like a really old version of Emacs. A nice window into the history of computer stuff!

Cool to see this idea in 1996. I think the main problem is that you have to write up / document everything for the Remembrance Agent to retrieve. The large majority of human knowledge stays locked to the brain, and sometimes our attempts at converting brain knowledge onto paper (“writing”) will leave omissions.

When i read the title i thought the idea may have been to engineer coincidence, which seems like a contradiction, but i wonder if is not impossible.

I think it's possible. One problem that I have with Remembrancer, as well as all approaches based on the same basic idea of 'show the user a list of similar items', is that this is fundamentally guaranteed to be mostly useless because the user will already remember many similar items and have deliberately omitted them. Sometimes you will forget, and then it can be quite helpful, sure, but for the most part, this retrieval strategy is inherently limited.

What you really get serendipity is from re-encountering things you don't know or have forgotten. And we know how to predict if you have forgotten something: spaced repetition and the forgetting curve! To manufacture serendipity, you simply track 'flashcards' of important things, but optimize for the opposite of the normal spaced repetition goal: you want to review only things you are predicted to have forgotten by now.

https://gwern.net/note/statistic#program-for-non-spaced-repe...

Then, as a Remembrancer agent, you make it show preferentially items that are similar but you have probably forgotten. (The Remembrancer agent can be treated as a fraction of a review, on the grounds that even just showing you the title briefly can trigger serendipity and if it didn't provoke any reaction from you, then that is evidence it is not useful and that title should be downweighted and show up less.)

You can definitely engineer the chance for interesting things to happen by surrounding yourself with interesting people in interesting places, and it's highly recommended imho to do so.

that's an interesting and fair point. what about the computability of coincidence?

"Engineer" was too strong of a word, IMO.

You can increase the odds, so that it is likely to happen more often. You can't engineer an instance of it. You can't compute it. If you could, it wouldn't be coincidence; it would be design!

Yeah, perhaps not engineer the time and moment in a predictable fashion, but instead just maximize your kismet per square meter.

People talked about this on HN earlier: https://news.ycombinator.com/item?id=25968751

The short end of it is something like: serendipity = work + telling people about the work

Similar to the system gwern mentioned in a sibling comment, I have implemented a "spaced inbox" system [1] that shows me old notes that I have written. The idea is to encourage me to keep adding to and refining ideas I have (but of course, the scheduling can be tweaked to only show notes that I am likely to have forgotten, as gwern suggests). Perhaps coincidentally, both my spaced inbox system and the Remembrance Agent in the OP are designed to work with Emacs (though mine does not force the user to use Emacs).

[1]: https://github.com/riceissa/spaced-inbox

Consider an alternative view, that coincidence is an immutable feature of reality that can be searched. In that case, you aren't necessarily engineering coincidence. Rather you are searching with the intent to align your own actions with the inherent coincident bias of the universe.

Reply on news.ycombinator.com