I feel like there has been a few takes on this (atuin desktop comes to mind[0] but apparently it's not a going concern anymore?) why hasn't it caught on?
For people who used these tools long-term: what held them back? UX/accessibility, or do people simply prefer keeping documents separate from the actual work?
we saw a bunch of issues with desktop, in no particular order
1. authoring runbooks is difficult, most people barely document their work let alone make it executable
2. keeping runbooks up to date is difficult, they are error-prone across systems
3. agents are now pretty good at a number of the tasks these ideas solve for, and are good at working around issues caused by docs becoming out of date/etc
all these tools are great for your own notes, but are very difficult to scale
Very happy to hear what you think of it. I've built it specifically for reviewing outcomes instead of impementations.
Humans define what correct looks like, agents implement and attach evidence (image, videos, etc.) that humans can review.
I've updated the landing page so it's clear that it's in beta and added a small "your data" page for now https://higherlevel.to/your-data
1. Ledge's bet is it shouldn't feel like authoring at all. I've always had markdown docs chock full of commands and docs so making them runnable was the next logical step for me.
2. True. Keeping any kind of documentation up to date is difficult
3. Also true. I'm hoping Ledge's built-in agent support will bridge that gap and allow agents to help keep the docs up-to-date
And yes, a "Team" notebook is a challenge. Technically anyone with SSH access can all share a Ledge notebook. But that probably doesn't get us all the way there.
Thanks again for providing your thoughts. This by no means is a solved problem but I'm interested to see where it goes!
To give a bit more discouraging advice to Ledge, these tldr insights have made me run circles to resolve. My experience is that one will start getting into build system and dependency mapping to make this sustainable. And then if one wants something more universal, one ends up building something like homebrew, which doesn’t even cover all OSs for good reason.
As you dig deep across system dependency and updates and configurations, there are crazy, crazy, crazy cascading bugs that popup from personal experience. As an example of how this comes up in practice, this is why many teams end up building on Electron as a common cross-platform bug fixer (Tauri has catching up to do although generally works well).
- Keep diagram source (eg Graphviz dot) with the document.
- Write software developer manuals that can reference code robustly as the code changes (eg include code snippets from source based on regex).
- Literate Coding / Reproducible Research pattern. Eg, org file explains and generates/builds/runs code and graphs/figures which then get included back into the document.
Some of the problems
- At some scale, one needs a DAG (eg make) or to make every command idempotent with fast no-op. Otherwise, each little change to a source block takes too long and there is a worry that something didn't update which should.
- At some scale, the document becomes the size of a library/package and it does not compose and/or I want to run the code outside of the document.
- The unenlightened around me do not use Emacs so an org file is a "me" file. In some ways this is a plus as it keeps others fingers out of my pie but it also means no way to share the baking.
That was in the days before LLMs. As ellieh's #3 points out, things are different now (we all know that). I now have an LLM externalize org-babel. The LLM maintains an org or LaTeX document describing bits of work, an external library/CLI which runs to produce content including putting numerical results into LaTeX macros, a Makefile or Snakemake to regenerate content and figures. When things are found to change prior understanding the LLM remakes a section of prose in the LaTeX document. I then write my own notes or another LaTeX document so that the trip through eyeballs to fingers on the keyboard assures I keep some level of understanding.
Likewise, in the software documentation goal, it's far better to give a good LLM access to the source and have it generate documentation targeting some learning goal with follow exploration via Q&A than it is to read some prepared document that assumes my goal. Software documentation is kind of a relic useful only for those people that have not yet taken up LLM tools.