I use Org Mode for personal logging/journalling | Hacker News Reader

I use Org Mode for personal logging/journalling

kevinkle.in

157 pointskkleindev48 comments

Org Mode is amazing! As an attorney I spend a lot of my time taking notes, tracking moving parts, and working with voluminous PDFs. I've tried a lot of different tools over the years (OneNote, etc.) for this purpose, but they all have serious shortcomings. I used to use Emacs in my previous life as a programmer, but had never used Org Mode before. I've got the following workflow:

- Org Mode (with org-superstar) for taking notes. Everything gets saved into a Microsoft OneDrive for sync between my laptop and desktop machine.

- Org tasks and org-agenda for project planning. Far less disruptive to your train of thought than busting out Microsoft Planner to remind yourself to follow-up on some line of research or whatever.

- pdf-tools for viewing PDFs. It's a bit kludgy (it renders PDF pages to PNGs on the fly because that's what Emacs can display), but it's somehow still less CPU-intensive than Acrobat DC or anything based on PDF.js. PDF-tools has fast incremental search, and a fast occurs mode for finding and highlighting all hits in the file.

- pdfgrep and pdfgrep-mode for searching across PDFs. Hits show up in a grep-mode buffer and you can click to jump to the exact hit. (It uses libpoppler under the hood to confine the search to the PDF text layer and show hits in logical page order rather than PDF file order, which can be arbitrary.)

- org-pdftools for creating links to specific pages of specific PDFs in Org Notes (e.g. noting that the testimony for a particular witness starts on page 758 of some 2,000-page trial transcript).

- Built-in Emacs features (windows, frames, buffers) make it easy to juggle between documents and compare them side-by-side. Emacs Bookmarks make it easy to save your place and jump back to where you left off. It's shocking how many notes tools get this wrong and only let you see one document or attachment at a time.

I'm kicking myself for not having discovered this earlier!

Do you have smooth scrolling in Emacs? How do you sync your work with your smartphone?

> Do you have smooth scrolling in Emacs?

Yes, but one of the smooth scrolling modes was too slow for me and I use `pixel-scroll-precision-mode` which requires Emacs 29.

> How do you sync your work with your smartphone?

I use syncthing for this and Orgzly, emacs in termux, and organice.

I bet most would prefer and could get away with just using organice.

Last time I checked organice didn't support offline editing on mobile. If that worked it would likely also be my weapon of choice!

Looks like it still doesn't:

https://github.com/200ok-ch/organice/issues/730

You might be able to run something with WebDAV support and use that?

Maybe sabre or caddy plus WebDAV plugin?

Orgzly work good enough with nextcloud/WebDav for me. Possibly easier to set up than syncthing for non-techies.

I use Mitsuharu Yamamoto's emacs-mac-port, which has smooth scrolling: https://bitbucket.org/mituharu/emacs-mac/src/master/README-m.... Due to the way Emacs works, pdf-tools can't do "continuous scroll" mode. This doesn't bother me much because I always have PDFs set to show the whole page anyway.

I don't sync Org notes with my smart phone. But the nice thing about Emacs (compared to most other notes tools) is that file attachments are in your file system. One Drive's sync backend is very good and smooth, so it's easy to grab a PDF if I want to attach it to an email or view it from my phone or tablet.

Interesting to see habits tracked using an org property drawer. I hadn't considered this before. Org itself has habit-tracking support https://orgmode.org/manual/Tracking-your-habits.html using :STYLE: habit.

I'm head-in org land. For habit-tracking, I found removing friction from the tracking process itself absolutely fundmental for longer term success (Atomic Habits, the book, corroborates). Like the author, I also needed mobile access for this. I built https://flathabits.com (powered by org) for this purpose (iOS only). In the end, I found I hardly used Emacs org mode to view or track my habits since the mobile app removed all the trakcing friction. Now I just get peace of mind knowing that I can access the plain text file if I want to.

ps. I also built another org mode app for iOS https://plainorg.com

Your work is contributing to the org-mode renaissance; users of the world appreciate you! Now if I could just get multi-platform sync working correctly (iCloud doesn't play nice with Linux)...

WebDAV is the fuuuuuuture.

Do you happen to know of an iOS solution that plugs into https://support.apple.com/en-gb/HT206481#thirdparty that you're happy with?

Hey, that's nice to hear. Have you seen some of the syncing alternatives? https://plainorg.com#cloud-providers

Mileage varies across iOS providers, but some users have reported being fairly happy with either WorkingCopy or Syncthing/MöbiusSync which would work on Linux. I'm on iCloud though.

I'll probably give MöbiusSync another shot; my first experience with it was heavy on battery usage and notifications, light on functionality.

More discussion in the space:

https://www.reddit.com/r/plainorg/comments/rerf1i/whats_the_...

https://www.reddit.com/r/plainorg/comments/qfrf87/please_sha...

There's a school of throught, prevalent at the last Emacs conference (See Karl Voit's talk), that Org is also a notation/markup. So-called "org-down" has been put forward with foundational semantics, headings, lists, emphasis and other basics.

https://karl-voit.at/2021/11/27/orgdown/

I'd actually love to see Gemini and HN adopt orgdown. Just sayin... :)

Thats the cool thing about org, its not just a great outliner tool and the best markdown, it also covers the full gamut of productivity and personal knowledge management functions (todos, calendaring, linking etc).

That said, what would it mean for HN to adopt orgdown?

It means one can use the orgdown syntax when writing comments.

In the meantime, there's https://github.com/tecosaur/emacs-everywhere.

It is. You can use org mode to write a scientific paper, personal wiki, blog posts, do literate programming, etc. I used to use Rmarkdown but I prefer using org now. It has better support for tables which is handy for printing out data frames.

I found out recentishly that GitHub supports org mode too.

Org Mode tries to be an outliner. Outliners are the right tool for the job from my experience (> 30 years) since you can create and use context to reduce redundancy and makes it easier to find and to understand things again retrospectively. I had a look at Org Mode many years ago when switching from Windows to a Linux laptop which meant that I needed an alternative for Ecco Pro. But I quickly reached the limits with Org Mode and a few other tools and finally developed my own tool, which has proven itself over more than a decade in daily use in large, complex projects. A few years ago I released it as open source: https://github.com/rochus-keller/crossline

What does your tool do that Org Mode doesn't? I.e. what are those limitations you ran into?

E.g. more typing, or you have to look to the screen when navigating; in Crossline - even more than in Ecco Pro - I can operate with shortcuts, but shortcuts optimized for an outliner, not just an editor with structured formating; Crossline allows me to look at the people I discuss with during the meeting and take structured notes at the same time without taking my hands off the keyboard and without looking to my laptop; of course one could build something like Crossline on top of Emacs (with limitations), but there is a point where you start to fight the fact that Emacs is an editor infrastructure, not a general purpose GUI toolkit; that's why I implemented Crossline in C++ using (modified) Qt.

I'm also a huge Ecco fan and definitely understand your motivation for creating Crossline, but the particular features you mention aren't really ones where org-mode is currently lacking, especially given org-roam.

Where I think org-mode is still somewhat lacking compared to Ecco is in being able to display the same item in multiple places in an outline and be able to edit any one of them (as well as the whole column UI for setting that up easily). But there are solid moves in that direction, e.g., the org-transclusion package is now pretty decent. There's also another more recent one whose name escapes me.

Crossline supports true "transclusion" links, so you can e.g. compose outlines from existing outline items with navigable forward and backward references; this is very useful e.g. for specification authoring.

I found a similar problem with Org mode when it comes to writing elisp code to handle org documents - it's almost all based on text parsing vs nodes. So simple automations I can program in a node based editor like the Leo Editor took a lot of effort in Org mode.

As an example, if you want the body (text) of a node, you have to get the substring from the beginning to the end. There wasn't (probably still isn't) an API to get the body. If you want to do a depth-first/breadth-first traversal of nodes, it's not as straightforward as one would think.

I hope someone comes up with a node based API for Org mode.

I implemented my own NoSql database for Crossline which supports an efficient storage and traversal of outline items (see https://github.com/rochus-keller/Udb). Even if there was a dedicated API in Emacs/Org-mode it's still file based text buffer interpretation and management behind the scenes.

Nah - the only pain would be in loading/saving. The idea would be that all this information would be in memory and updated live. So when you use node based APIs, it doesn't need to do any parsing. It would have a node object, and the body would be an attribute, and subnodes would be links to other nodes, etc. Only when saving/loading would it then need to convert (serialize) to a text representation.

This is how the Leo editor does it - it's in Python and it can handle a fairly large number of nodes before any slowdown is noticeable.

If you're trying to do non-parsing automations in org, take a look at vulpea. It's roughly the same engine that powers the current version of org-roam. A good example application of this is the vino wine database: https://github.com/d12frosted/vino

Thanks, but is there something like that for plain org? I use org-roam for a few limited use cases. Most of my org stuff is not related to notes, and I don't put it in org-roam.

vulpea is designed to be usable under plain org; it's a separate package from org-roam.

Your comment is probably true, org-mode vs X features, for any tool X, but org mode (ie all of emacs) is a tool construction kit, not an especially good tool itself.

I, too, was a fan of Ecco in the 90s, but it was discontinued in 1997.

OrgMode wasn't even born until 2003, so my guess is that you made your choice about Org very, very early on.

I cautiously suggest that maybe Org does more now than you realize. Your tool seems . . . well, very dated-looking to start.

No, I worked with Ecco until about 2010 and then migrated all databases to Crossline; even though the original company didn't continue development of Ecco Pro there were (and still are) free installers on the Web and even people providing patches. My evaluation of Ecco Pro alternatives had it's peak in 2009 as far as I remember. Org-mode was already established. Of course I was biased as a long-time Ecco Pro user and looked for something with similar or even better features; Crossline can do things I missed in Ecco Pro and doesn't do things I rarely or never used in Ecco Pro; and I didn't spend a second in making it fancy looking, but a lot of time in making it as efficient, usable and powerful as possible; and it's definitely not suited for mobile devices, which is no problem for me since I need a laptop with a track-point to be able to work efficiently.

Org mode does not "try to be an outliner", it does much more than that. It can do that in 2-3 key presses, which fold the content under each heading.

> CrossLine is an outliner with sophisticated cross-link capabilities in the tradition of the well-respected Ecco Pro.

This is the minimal description in the repository you link to. It does not contain anything, that Org mode does not bring to the table. Org more has sophisticated inter-document linking as well as citations and footnotes. All one needs to write a scientific paper in it and export to tex/latex.

What features exactly is org mode missing for you? I know you wrote, that you tested 10 years ago. Perhaps the picture has changed a lot since then.

Also I understand, that ones own tool often is like a second skin, perfectly matching, what one needs.

I've been using org mode to maintain a personal work log for around ten years.

General structure is a file per month, a toplevel outline heading per day, and a global hotkey that takes me to 'today' with a single button. I use a private git repository to archive it to a secure place and replicate across different machines if needed. (Not common.)

Code here:

https://github.com/mschaef/.emacs.d/blob/master/lisp/orglog....

It's worked well, but generally speaking the key to logging is less the code/system and much more the discipline to write every day. I'm hit or miss on that front, but still find it useful to both formalize my thoughts on things into english and also occasionally go back to look at previous things I've done. (Oh, and it makes developing lists of accomplishments for annual reviews an easy thing.)

I did a lot of time logging for orgmode. It was great if ones task are all done on a single computer.

I found trying to synchronise across devices tedious and made me stop logging.

Just log into a remote shell and execute emacs-org-mode from there. Like, remote shells are free 2022, no? I agree that if all the devices can't generalize to one source then it isn't as rewarding.

Cool idea! I do like to be able to work on it offline, e.g. on planes, though.

Have you tried using Syncthing? It's open source, free as in beer, and should fit your usecase well. I'm using it for my Obsidian notes, and it's pretty much frictionless.

My current setup is also slightly dissatisfying in that regard. What's nice (or rather a consolation) about orgzly - the mobile app I use - is that upon conflict, it explicitly offers to either choose the remote or the local version. At first I was worried it would simply attempt to always overwrite based on the local version.

It combines even more nicely with Syncthing, which creates a copy on conflict, which can then easily be merged with ediff-files once I get back to a computer.

Started using org-mode a few months ago. I keep my notes and todo list in a self-hosted git repository.

On desktop I use Magit to push/pull changes.

On Android I cloned the repository in Termux and pointed Orgzly to it. This wasn't absolutely trivial, can provide details if needed. Also created some convenience scripts for git operations (commit-and-push, pull, check status). Now I can push/pull changes with singe-letter commands in Termux.

I find the friction to be low enough. Using git is also very convenient to resolve any conflicts in case I forget to push or pull some changes.

I just keep my files in Dropbox and haven't had a problem yet.

I tried using Org and Org-Roam for a while, but I think I gave up on it. The overhead of constantly updating and fiddling to make your config comply with the updates didn't seem worth it. The alternative would be to just stay on an old version, but that goes against my principles of running a rolling release distribution. And hell, if you decided to update after a year or two, you could probably start over with most of your configuration, rather than trying to understand the updates.

I rarely need to update my config for Org - they rarely break compatibility. Perhaps once every 2-3 years.

Org roam - that's not a mature project and it frequently breaks. If the current Roam is doing what you need, do not let it upgrade automatically.

I'm a huge Emacs user but I totally suck at org-mode. Now I do use it but mostly for the outlining functionality: I love it that I can quickly collapse/expand by "x levels" (for example only level 1 titles, up to level 2 titles, etc.). But most of all I "search don't sort": it's very common that I "ripgrep" (from Emacs of course) inside my .org files. Even only using like that, without knowing much about it, it's still very useful.

I do version everything under Git so when I switch machines/countries my latest notes are one (ma)git command away.

> I wrote a script to apply that template to every week and month of the year. Hence $ ls ~/org/weeks/ returns the following:

You don't need this - you can use Org's capture templates with datetree type. It will then have it all in one file with appropriate headings.

Tip I didn't realize for years: Sometimes you forget to log it on the day intended (e.g. for the post-month you log it 5 days after the new month when you should have done it on the 1st day). Prefix your capture command with C-1, and it will prompt you for the date to file it as.

Interesting to see as an entire submission rather than just every other comment.

Reply on news.ycombinator.com