Using VimWiki as a distributed, encrypted lab notebook for programming | Hacker News Reader

Using VimWiki as a distributed, encrypted lab notebook for programming

stochasticgeometry.ie

66 pointsmarkdennehy34 comments

I have a similar system for a lab notebook / to do list, but without encryption. I use a light weight wiki, Zim http://zim-wiki.org, and setup the directory in my Dropbox folder. That way I'm able to use the full wiki on my desktop, laptop, and work desktop. Because the pages are plaintext with simple wiki markup, I can also view and edit them on my phone.

Zim is also nice for me in supporting LaTeX equations (via a plugin) and being written in Python, I've made a couple of custom tweeks.

Yup, and using dropbox was something I was looking at for vimwiki as well, but the whole "stored in the cloud" thing and the whole "NDA, sensitive data" thing didn't mix (especially in the EU, where you have a whole host of other things to worry about).

A difficulty I have with wikis is I find that tree structure data of this sort gets stale easily, and requires a lot of tending to avoid this.

A tool ppl might find useful that uses a flat structure: saga. All it does is creates directories in a way so they will always sort in chronological order.

Example:

    $ saga python pattern socket client
    /home/cturner/saga/20121124.myhostname.myusername.ac.enter.some.terms
    $

This turned into a long reply, so I blog'd the rest.

    http://trogrd.tumblr.com/post/36378631988/saga-revisited

Correction to editing error in parent. The resulting directory would be called /home/cturner/saga/20121124.myhostname.myusername.ac.python.pattern.socket.client

(thanks kartik)

Fark. Apparently landing on HN at 1800 on a friday is a fast way to kill your server :D Think it's fixed now... though it might not be a speed demon...

Not to take away from any of the ideas, but (I believe) there is still no electronic substitute for a dated, numbered-pages paper notebook to establish that you did something and when you did it (for legal purposes).

Love these: http://www.amazon.com/Computation-Inches-Numbered-Quadrille-...

Absolutely; and I keep mine still for doodles that become patent ideas (signed and dated), for taking cornell notes of meetings and phone calls, and that kind of thing. I prefer these though: http://www.leuchtturm1917.com/en/content/medium-notebook (and the dotted paper is more useful than the squared). Tried moleskins, tried Paper Blanks, tried a few others, but the 1917s are the best I've found...

...but they've become my first draft, rather than my only notebook. It's just too easy to grab a chunk of an email or a graph's png or a code snippet and lob it into the vimwiki; and it's where I work up most of the stuff that goes into papers or patents.

I'm a big fan of Doane Paper. Lines AND Squares. http://doanepaper.com/

I know some people who used an electronic system which printed out a barcode encoding a cryptographic hash of each entry, which could then be stuck in a paper notebook with numbered pages, and dated and witnessed.

This combined a legally admissible chronological record in physical form, with all the benefits of electronic records.

I haven't even kept a log like this, how useful do you find it? Is it similar to making a lot of small commits that you can search against?

It's similar I suppose, but you'd have to be committing every 30 minutes or so (maybe your memory's better, but I find if you asked me to summarise what I did in the last 30 minutes I can fairly quickly; if you asked me to summarise my day, I'd forget half the stuff I did). And you'd have to be always working on the one code base (I've worked in places where you wouldn't have been doing that) or else you'd have entries scattered all over the place.

But yes, if you did those things, you could have that log in the commit logs, but it wouldn't be a fantastic resource - it's more useful to me to have that in a local file here, with just my entries (as opposed to having to extract my entries from the entire team's), in grep-able plaintext.

And you'd also be missing the notebook part of the picture; the stuff in there shouldn't be in commit logs (you're talking about notes from meetings, design notes, patent notes, paper notes, that sort of thing).

I use org-mode for 'partial' literate programming. It's partial since I only note down the general structure of c++/java classes as a sketch, not actual codes, except for Lisp. I occasionally push the files to Gollum, a git-based wiki supporting org files for mobile reviews.

I've seen some folks do that, and hats off to them; but I learnt vi before emacs, and despite trying to learn emacs a few times over the years, I just never did get to the point where I'd feel comfortable using it for work. So org-mode was always out for me. But it works well for others...

This is interesting; my site provides something sort of similar, but with JIRA integration. I've been considering a pivot where the data is stored on the local server. Rather than using JIRA credentials, you'd define your own TODO list, and log 15-30 minute intervals with notes on what you've done. The hard part is creating the reporting faculties; there's lots of those for JIRA already.

If anyone's interested in something like this (structured time-tracking and task-based note taking), drop me an email and I'll let you know when I get to that point.

It sounds interesting, but one of the things I didn't want was something tied to a specific system. This is a notebook that goes across several jobs - and depending on where you work, you could be using JIRA, or Mantis, or bugtac or something homegrown, or someething horribly legacy. So I didn't want it tied to anything specific, but to be a standalone tool.

Mind you, if you were working a lot with JIRA, I could see the appeal...

TLDR: Notational Velocity to vimwiki.

I was using NV, but the Find-Next feature stopped working from OSX Lion. I then renamed the file extensions to ".wiki". I used some perl scripts to remove the spaces in file names and camel-case them. Used "ls" to create an index page with links to each note. Now i can access my notes both through vim (using vimwiki) and through NV.

vimgrep and the ack plugin are good replacements for NV's search.

There was a discussion on HN a while back on maintaining a programming journal except this one advocated a pen-and-paper approach. Might be a good place to get ideas on the what and how-to aspects of maintaining a journal.

http://news.ycombinator.com/item?id=4448361

+1 for VimWiki; I don't use it like this author, but it is one of my favorite Vim plugins.

How does the VimWiki plugin improve it?

I have used vim for journaling and as a wiki, but I only used standard vim features. For example "gf" is practically for a flat file wiki-it opens the file named after word currently under the cursor.

Vimwiki, amongst other things (and I'm still learning) supports not just its own wiki syntax, but also markdown and mediawiki (there's an option to be set and it has to be after v2.0); it has a decent diary system with automatic index page generation; it can automatically convert the entire wiki+diary into html if you want that; and it has a few other nifty things like checkboxes for to-do lists and the like. Worth taking a read through the features part of the project website at least.

Off the top of my head, some light formatting support, and simplified "link" navigation.

Taking your example - Yes, I know of "gf" and "ctrl-o". VimWiki changes that to "enter" and "backspace", and requires no extra steps to visit filenames with spaces.

Not wildly ground-breaking, of course, but I like the end result.

What are "cambridge format notes?" Couldn't find anything obviously that.

Ah, feic. It's Cornell format notes - for some reason I keep calling it Cambridge, I don't know why. Google Cornell Note Taking System, and learn it; if you ever have to take notes at a conference, seminar, lecture, meeting, phone call or anything else where people are talking, it's the best method around.

This right here made the whole month of reading HN worthwhile. Thank you.

My pleasure. I just wish someone had shown me the cornell system before year one of college :(

vimwiki + dropbox user here too. I was a big time OneNote user until I needed my notes on all the major platforms at once. I realized that I would much rather keep them in a text-based format, rather than in a proprietary encoding with no API, even though I think OneNote is a fantastic tool.

So far it's worked fine for me, even though I wish it supported Markdown out of the box (there's a plugin for it somewhere), rather thank its custom markup. It's somewhat painful to have to remember yet another syntax to achieve the same exact thing.

vimwiki does support markdown out of the box as of v2.0, though you have to set an option for it...

let g:vimwiki_list = [{'path': '~/my_site/', \ 'syntax': 'markdown', 'ext': '.md'}]

I've been using vimwiki for a while now and had no idea about this. Thank you so much!

(Now if only I could figure out how to get Vim Touch on Android to correctly format vimwiki pages...)

Wonderful, thanks for letting me know! Goes to show how long I've been using it for! Can't wait to convert everything to md.

I've always found hard to write scheduled log entries. I feel more confortable with task driven logs, think about a log made by commits of a repository of my work life.

I think mercurial is overkill for this - much simpler to just use dropbox.

Dropbox lacks major features if you try to label it a DVCS, and it's nowhere near as flexible. Most notably, it has ZERO merging support - conflicts result in silently duplicated files (with an added timestamp).

Exactly, and you're also assuming I could trust the data to dropbox. Sorry, but no, I'll trust photos and nonsensitive personal data to it without too much worry, but NDAs are legal contracts and the whole cloud storage thing and EU law are murky enough to make me just step away from the keyboard. McGarr's (an Irish solicitor) has a good summary here: http://www.mcgarrsolicitors.ie/2009/09/11/cloud-computing-eu...

Fair enough with respect to NDAs in your case, but they don't really apply in mine. As for it missing features of a fully-fledged DVCS, meh. What you're talking about is basically a glorified TODO list, which in most cases you'll be the only one updating. Furthermore, you'll usually be updating it from one place at a time. On the rare occasions I get a conflict, it's usually a simple matter to resolve. I do use Mercurial for managing other stuff like environment configs and program settings though.

Reply on news.ycombinator.com