Using a logbook to improve your programming | Hacker News Reader

Using a logbook to improve your programming

routley.io

295 pointsjmlr74 comments

This article seems lacking some examples. I have found that such ideas are nice in theory, but when it comes to practice, they tend to fall short.

So the author says that it's a good idea to write logbooks, and that it will help. But no mention on how it would help, and what would you actually do with them after you write them up.

A few concrete examples would have been nice.

If the idea is to help you solve the problem more efficiently, why should it be formalized? as opposed to scribbling notes on a notepad that you throw away at the end of the day.

Advice of this sort is tough. I really really want something prescriptive.

I suspect it is ninety percent confirmation bias. Looking back at successes and assuming something in common of all of them had to be contributive.

Not claiming it is all luck. Just saying that outside signals usually bear more meaning than given credit.

Well, I use this logbook approach, in a text editor with persistent undo (which allows a few tricks, like emulating folding by deletion and relying on undo history to unfold). I approximately follow this algorithm:

- Consider the high-level goal; can it be completed immediately? if yes, do it, otherwise break it into smaller parts.

- First part is usually some kind of research. If it's a bug, it'll be a reproduction. If it's a well-specified feature or a refactoring, it'll be locating all areas of code that need to be updated. If it's design, it'll be an initial sketch.

- Subsequent parts depend on the task at hand. For a bug, I generally try and work through to a minimal set of reproduction steps. For a feature, it'll be a generated list of tasks that came out of the research step.

- When working on any given task, I add sub-entries in the form of a dialectic. For a bug reduction, I come up with a hypothesis, then try and prove or disprove it by testing. For design, I propose one approach, then consider the implications; then another approach, and so on. For a feature, tasks generally get added for any "todos" that crop up as I implement things, to ensure I complete everything.

I essentially record salient details of my internal monologue to ensure that I can pick up context again easily, and generally fit more into my working memory by offloading it into text.

I reckon it made me perhaps twice as effective an engineer, particularly in distraction-prone environments. It's also made me a lot more confident that I'm not relying too much on memory as I get older - not that I think I forget much now, but I believe I won't be very dependent on a large working set.

Emacs Org mode works wonderfully in a scenario like this. Consider trying it.

Better yet, try org-journal, which automates the one-file-per-day part, integrates with the calendar, and adds searching.

Do you have any ideas on how to adapt this to someone who doesn't have a "present" internal monologue?

If I concentrate, I can sense something akin to the typical "internal monologue", but it isn't present enough to be copied down, and even when it is it tends to move so fast that I only really get the conclusion.

Interesting, when you think about something you don't ever hear yourself talking it out inside your head?

I can hear small parts of it now and then, and I can definitely 'hear' myself read and write text, but most thought doesn't pass through my internal monologue. I can't tell if it's because it's going too fast (Which is certainly possible), or because of some other reason.

Could be you're a more visual thinker?

Imagine you're explaining what you were doing to a programmer pairing with you, or to a junior dev, or just someone who is smart but lacks context.

When I started coding, as a teenager, I'd pace around the room explaining things to myself almost as if I was giving a lecture, so it's fairly baked into my thinking process. If you assert something, laying a statement out there, the process of hearing it can cause you to start to think other things: is it actually well-justified, are the assumptions behind it solid; what are the consequences and implications of the statement; all the things you might think if you heard someone else say something, and you're listening critically and intently.

> Do you have any ideas on how to adapt this to someone who doesn't have a "present" internal monologue?

If you use a text editor you don't need to write everything down right away. You can just dump a keyword or a snippet that describes or is associated with what you're doing, note your intent and/or your goal, and as you work on your task keep updating your notes whenever you get the time.

"It provides a framework for solving hard problems by encouraging you to break them down into a series of smaller ones.

"It helps you focus on the task at hand by providing immediate context on what you’re doing. If you forget or become distracted, you can quickly get back to your train of thought.

"It helps you learn quickly. You can observe your method for problem solving, see what works and what doesn’t and make improvements."

Saying "no mention on how it would help, and what you would do with them after you write them up" feels inaccurate to me (although the observation about the lack of specific examples is correct).

> Saying "no mention on how it would help, and what you would do with them after you write them up" feels inaccurate to me (although the observation about the lack of specific examples is correct).

To me it is an accurate description, as the author focuses on what he believes are the results while not presenting any tangible example or framework on how to get them.

In short, without any information on the "how" this blog entry boils down to a page and a half where the author states that he believes that writing down things he is doing in no specific way and following no specific method may help him in some way. Well, good for him.

It works. There's even a psychotherapy method called "problem-solving therapy"; all it does is that it formalizes the process of finding a solution to a problem: define the problem (what you have and what you want to have), outline possible approaches, evaluate, choose. I myself use something like that in my work (software development) with a text file and I can attest that following the steps often helps me to find or choose a different solution than what initially popped up in my head.

It doesn't solve all problems, e.g. a complex problem that is hard to understand are not a good fit; if I notice that what I write is lengthy or that I keep returning and restating it, then this is that kind of problem; here I normally leave the desk and try to sketch it on paper.

Very seldom I look through the file to find why I've made a particular choice; I don't think logbooks are of any use here, it's better to document the design elsewhere, closer to the product. I'd say logbooks are write-only (although they may be a source for the docs).

Surprised no one has mentioned doing this in a physical notebook with a pen. I started doing what the author describes a long time ago to document bugs to come back to later. I soon realized it's just a good way to keep my thoughts organized and be more productive. I tried doing something similar with a text editor but grew tired switching screens or tabs. A notebook is also a bit more accessible than a laptop when you leave your desk. During lunch I can read through my notes.

I've done this with a notebook for a long time and it is indeed helpful, especially if I didn't get enough sleep the night before or didn't start my day with a run and I find myself more distractable.

I actually I mentioned this 10 hours ago. So no need to be suprised ;)

I use a physical notebook to record my work too, and I find it very effective. If I sit through a meeting and write nothing down I will inevitably forget everything about the meeting, writing it down helps me to remember it. You can also go back and find out what was discussed before - a surprising number of meetings just retread things which were talked about or even decided upon previously.

I use: Oxford Black n' Red A5 Matt Casebound Hardback Notebook, and UM-153S Signo Impact Gel Pens. I get through one of those notebooks per year on average.

Edit: https://rwmj.wordpress.com/2017/11/24/keeping-a-work-noteboo...

Most likely you love high quality notebooks and writing instruments, and you probably have neat handwriting.

I rather have files that I can understand later, and easily grep as a bonus...

(I do admit that crossing done tasks on a piece of paper feels very good.)

I like to think of blank pages as mini whiteboards. With a multifunction copier i can print out lined, grid paper or specialty layouts and whip it through the scanner once im done with it keeping a digital copy. 3USD buys me 500 pages, and with a laser printer [250USD], toner refill kit, etc. that comes out to less than a cent a page, or 3 cents if I amortize the printer over 10k pages (assuming it breaks once it hits its one month maximum duty cycle, which is stupid conservative).

That plus a nice fountain pen [Lamy Safari 18.7USD], ink [J. Herbin Perle Noir 7.2USD/30ml], mechanical pencil [Lamy Safari 12USD], pencil leads [Uni Nano Dia 0.9USD/20leads], technical pen [Rotring Isograph 16.3USD], and I've got a nice little setup.

“I like to think of blank pages as mini whiteboards.”

This. I never enjoyed drawing diagrams in Inkscape, though my boss swears by it.

I’d much rather draw with pens on paper and then digitise by scanning.

I could see your bosses workflow working as long as he has a library of common elements. For graphs I tend to use yED, which is nice. You know how LaTeX has the mantra that you input the data, and it worries about the layout for you? yED is similar, you add nodes, connect them, and then you chose from one of two dozen layouting algorithms, like "hierarchical", "swimlane", or even "family tree".

I would love for a decent midpoint solution, ie, a way to digitize my notes. A livescribe would be nice, but the issue is that my handwriting is...not too great. I can understand it,certainly, but I'm not sure it would escape OCR unscathed.

Unless you have a good tablet, like the most recent Surface Pro or iPad. Writing on them feels very good, almost no noticeable lag. That's what finally replaced physical note-taking for me.

> I tried doing something similar with a text editor but grew tired switching screens or tabs.

I've found that using a text editor that supports code folding, and following a markup language that enables the text editor to automatically set fold points, helps navigate log entries more efficiently.

I also use a notebook to jot down tasks, but when a job requires a high number of different unrelated small tasks that are completed rather quickly I've found that a physical notebook actually makes the process very inefficient.

Based on a some of the comments so far I think people are glossing over the key elements of the logbook. This isn't simply an activity log.

The first step is to describe the nature of the problem as well your planned solution before beginning. I think this can be a pretty powerful practice in being mindful.

All to often I find myself jumping into a half-baked solution while at the same time having only half-understood the problem.

>All to often I find myself jumping into a half-baked solution while at the same time having only half-understood the problem.

This is what kills me in tech interviews. Unfortunately its hard to calm down and think quietly when you are being hounded by the interviewer to "think out loud".

I did not go to college/university for engineering, but just hearing the tools of 'real' engineers is great.

I've been opening up Jekyll in the morning as the start of my day. I write the blog post for the work I'm about to do. I've noticed that I'm getting better at explaining the problem clearly and the solution. When there is a paragraph that feels out of place I yank it into a different file immediately to get it out of the current flow of the post. The paragraph felt out of place because it was probably a distraction from the problem Im trying to solve today. Also, the great thing about this technique is that I have a blog post at the end of the work day that I can look back on and say, "Damn, I shipped something" if I didn't feel like I shipped anything else (common problem in our field of software developers)

Keeping a journal is particularly useful to resume your work.

Before lunch, or a meeting, or end of day, or when someone interrupts me and needs something, or any form of break, I "serialize" what I am working on in the journal, and then I read it when I am coming back.

In this way I can keep my mind clear and focus in the next activity.

It is also useful for standup meetings, retrospective meetings, etc... when you often need to describe what you did yesterday or this week.

Heh funny, I started writing a "labbook" like this ~6 months ago with markdown writer in atom for every day. I have a background as a Lab technician where I was used to keep a daily detailled labbook so I didn't have to develop that habit..

I like to structure my programming lab book in a similar way, for every task/ bug/ project I try to jot down a description, hypothesis, expected outcome and document my progress with code snippets / outputs / plots. I also keep a Todo list at the top to visualize my current workload. It really helps if you have to come back to a problem later on or just to get a good overview of your current task. I feel lik this also works nicely in conjuncture with GTD to increase my productivity.

I believe many of us already do what the author suggests, maybe in a slightly different way.

Here's how I use my own flavor:

1) I create a GOAL. The GOAL is my ultimate big picture, which will give me immense happiness. So, the most recent example is, I wanted to build an E-Commerce application.

2) I map out the PATHS to achieving my GOAL. So, what I consider PATHS is:

a) What programming language should I use? b) Which APIs should I use, Paypal or Stripe to accept payments[1]? and so on...

3) I follow a flavor of DDD[2] to come up with the important modules I need to have for a functioning E-Commerce project.

Examples: I will need a Order Management module, I will need a Marketing module (for sending emails, running campaigns). I will need a CMS module to host product pages.

4) For each of the modules, I will create an issue on Github and add a checklist[3] of the stuff I need to get done.

5) Everytime I make a commit, I will reference this issue and the tasks along with it. Eg. "Removed Paypal #12"

6) At the end of the week, I will review the commits and evaluate whether this particular method I used was worth it. A good use case would be experimenting with a library and comparing it with another - Eg. Bootstrap vs Semantic UI. I will use the learnings from this workflow in my future projects.

7) I will also see if something can be automated. For example, for my current E-Commerce project, I use Phoenix/Elixir. It has it's own set of scaffold generators, but they're opinionated (they use Bootstrap). So, for every project I got tired of replacing bootstrap and customizing the generated scaffolds. So, I wrote a custom generator that drastically saves me a lot of time for large projects.

Hope this helps :)

[1] Not sure why anyone would use PayPal in this day and age tho.

[2] https://stackoverflow.com/questions/1222392/can-someone-expl...

[3] https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-...

looks a lot like "ends ways means" approach

http://www.dtic.mil/dtic/tr/fulltext/u2/a222381.pdf

Somehow got me thinking about impact maps again: https://www.impactmapping.org/drawing.html

Maybe I go against the common thought here, but for me I think it is completely useless. Whenever I start working on a problem I obviously have an approach in my mind, but very often I find out that is the wrong one. And I can realise it only after I start working on the actual problem and solving it. At a certain point I will find some obstacle that will make me rethink the whole approach. For this reason I can't really see the added value in spending more time before actually tackling a problem once you have a plan in your head. The most important thing to do when solving problems is to be prepared to change your point of view multiple times and to attack the problems from several different directions. I'm sorry but I really can't see how writing in stone your approach before you fully understand the problem can help you in any way...

The way I interpreted it was basically an iterated Feynman Algorithm[1]:

1. Write down the problem

2. Think real hard

3. Write down the solution

4. Write down how well it worked

5. GOTO 2

I do something similar using org-mode, and it's useful when, after some time has passed and I need to do the same thing again, I can go back to my notes and not start from scratch again.

[1] http://wiki.c2.com/?FeynmanAlgorithm

I've found keeping a software-engineering logbook very useful at work.

Recommend the vim-wiki plugin for this https://github.com/vimwiki/vimwiki; in any vim-window the logbook for today is accessible via [leader-W-leader-W].

or maybe http://orgmode.org/ if you are using emacs. I do not know how awesome orgmode was, and I was missing it. plus you can export it even to latex if you wanna track your research and write your thesis.

I find this interesting. Mainly because, I've been trying to implement similiar strategy on my daily works to fight off low productivity hours and really figure out where all my time is going. One of the things I've been doing is noting down the time and date when I set to do things. Seeing something similar presented in the post with more formal and clean approach made me happy - as in being on the right track.

But than again, there was TED talk of all this organization is ultimately shifting the energy from actual. Which led me reducing number of 'productivity' application and sticking with a few.

Wunderlist Onenote Evernote - mainly for the phone.

Good idea, but as others noted it was a bit light on details.

Some more concrete suggestions from what appears to be the university of Idaho's Electrical Engineering department can be found here:

http://www.ee.uidaho.edu/ee/power/jlaw/COURSES/CAPSTONE/F05/...

What the author says about aliases in the bashrc being evaluated only once when the bashrc is run when the shell session is opened is wrong.

Go ahead and try this:

Put,

    alias lb='vim ~/logbook/$(date "+%s")'

in your bashrc.

Source the bashrc.

Run the lb command. It creates a new file with a different epoch timestamp each time. This `lb` command doesn't need to be a bash function. It can be an alias and work just fine.

Maybe how this works in zsh is different...

Zsh is different, although I'd be surprised if this wasn't the Bash behavior.

It doesn't make sense. If you do

    alias e=vim

it's not like Vim opens when your shell starts.

The only way to have a "precomputed" alias is to get the quoting wrong, and leave $() in double quotes instead of single quotes, meaning it will be evaluated before the alias is even assigned.

Their function uses single quotes on format string, so I'll bet they tried it like this:

   alias lb="vim ~/logbook/$(date '+%Y-%m-%d').md"

..which would do what was described.

Seeing as the author is using vim he could just have a command within vim.

  :exec ':e ~/logbook/' . strftime("%Y-%m-%d.md")

Emacs org-mode is great for this.

As is vimwiki, for those on the other side of the fence.

I've been using org-mode for about ten years to document moment-to moment progress on tasks. I'll give a basic summary of the problem, copy in significant commands and results as I run them, and ask the next questions to drive the next steps.

Some features that make Org useful for this:

- I'm clocked into the task so I can always jump to it easily, even as I hop around other emacs buffers

- supports code snippets with formatting in any language

- it's easy to add timestamped notes or write in free form

- can export the subtree to HTML or Markdown or Latex/PDF, to share with others

- GitLab will apply basic Org formatting (as with markdown), so I can share them directly just by pushing

- supports regex search across all agenda files, regardless of where they are. This is nice if, say, I know I had a similar issue with e.g. some docker command but don't remember which project it was on

i'd be interested in learning more about this and maybe seeing some real-worldish examples.

Not the OP but I use org-mode to write articles for my website and also to manage my dotfiles, this-- [http://abhirag.in/articles/org/i3_setup.html] is just an org-mode file exported to html, you can also see the literate dotfiles on that page, they have been generated using org-babel. I am also trying to setup a proper literate programming workflow using org-babel but that is still a work in progress. For a glimpse of what org-mode supports you can look at this this article of mine -- [http://abhirag.in/articles/org/custom_org_theme.html]. Hopefully these examples helped but I have only scratched the surface here, have a look at-- [http://orgmode.org/worg/org-tutorials/] and other stuff on the official website to find more information :)

Org is already 10 years old? It seems like yesterday that I was using Muse Mode. Then, seemingly overnight the muse discussion lists went very quiet because everyone switched to Org. I still have a bunch of notes in files that was created in Muse.

Shameless plug: org-journal automatically creates one file per day, integrates with the calendar, implements searching within date ranges, and optionally rolls over TODO items to the current day.

This is similar to what I've been doing on and off for nearly 5 years now: https://gist.github.com/sent-hil/3444793, except on paper.

https://news.ycombinator.com/item?id=4448361 has some good tips and tricks.

I have similar ideas. At the beginning (20 years ago) I use windows notepad.exe. It has a trick that if you put “.LOG” on the top of a txt file, every time you open it with notepad, it appends current date to it. Later on I switched to Evernote, then Apple Notes.app and settled. Recently I added a service with Automator to allow me insert current date to a note. Quite convenient.

F5 inserts the current date / time in Notepad.

I don’t keep a log but I use pen a paper to create diagrams which visualize my problem and potential solutions. It has been very helpful and good excuse to rest my eyes from the monitor. I don’t care about dates cause I like to work in a chaotic kind of way and I don’t like planning ahead I prefer exploration and critical thinking during execution.

Might as well auto generate a header for the file...

  function lb() {
    today=$(date '+%Y-%m-%d')
    fp="/home/you/logbook/${today}.md"
    if [ ! -f $fp ]; then
        echo "# ${today}" > $fp
    fi
    vim $fp
  }

I've been using Typora to make my todo lists recently. It's a markdown editor. Super simple to type, and you can really got some professional looking layout, with headings and titles quickly too. Recently they built in a file browser. Open one file in a directory, and it will show you all the files in side panel. Which in my case are just todo lists for day of the week. But it could be any type of notes. Highly recommend it. For Example typing ###Title would give a large title typing - [ ] will give you a checkbox, and so on.

My "bash function" adds just a few more lines, so as to git add/commit/push after closing the editor, and adding a good title by default:

----

journalfile=$(date --utc +%Y%m%d-%u.md);

if [[ ! -f $journalfile ]]; then

echo "# "$(date "+%A, %e %B, %Y") >> $journalfile;

git add $journalfile;

else

echo "File already exists: $journalfile";

fi;

vim $journalfile;

git commit -m "Changes in journal file $journalfile" $journalfile;

git push origin master;

----

Source: https://github.com/samuell/mdnote/blob/master/editnewjournal...

I like the idea, i just think MD files might be too limited. I really came to enjoy Onenote because it is so easy to put unstructured data, images, links, videos and even draw on top of it all with a stylus but i didn't use it for every task i do. Will try that in the future as i often have issues resuming old tasks or remembering some details to a problem in the past.

This is a nice and simple! Usually I keep a logbook separated for each project but it looks more like a diary where I record failed approaches and new tries to solve a problem.

For more structured documentation, I keep separated text files for each solution that I can easily find if I need it later.

EDIT: Also, scratching your plan with pencil and paper before implementing it helps a lot.

I keep a log of things I don't really know but should invest some time in to learn it properly as I make use of it. That's quite useful, not so much the log, but the forced awareness of making a mental note. I've also been thinking of keeping a log (repo with notes) of the incremental experiments I do when learning things.

In between commit messages, bugs and design docs, I'm not sure what I'd put in a log book.

- The logic/design of parts of the code base which you might newly encounter

- What work was done for the day, yes you might use something like Jira but a journal can be also good at the same time

- New things you learn.

etc etc etc

Each to their own of course but it seems to me, there's better places for all of that. If there's logic or design that's complicated and not properly commented or documented, then that's worth doing and sharing with the team.

Work done for the day for me is typically captured in commits or bug updates - that way my colleagues can easily take over if I win the lottery or are suddenly taken ill.

And stuff I learn is learned, so I don't need to write that down :-)

Working in a team, shared information is critical and logbooks just aren't the best way to achieve that.

> If there's logic or design that's complicated and not properly commented or documented, then that's worth doing and sharing with the team

It's the meta also, it is capturing the models mentally and ideas and notes. This reduces cognitive load, and actually by making notes increases the chances of you remember something.

> And stuff I learn is learned, so I don't need to write that down :-)

Hey if thats the case then fine, but generally memory deteriorates, recall becomes more difficult and the details change.

> Working in a team, shared information is critical and logbooks just aren't the best way to achieve that.

Log books are not generally for the team, they are for you.

But as you say each to their own.

This does not have to be "either-or". As we see an increased focus on teamwork this could evolve into a shared (electronic) team logbook.

I think maybe documenting architectural decisions might make sense: http://thinkrelevance.com/blog/2011/11/15/documenting-archit...

I find writing documentation as I go to be a particularly good form of this. I start before I fully understand how I'm going to solve something, and only check the documentation in once it's correct.

I’ve been exploring keeping a logbook for awhile. Some apps I recommend: Quiver, Boostnote, and jrnl.sh

All of them have open data formats so you’re not locked in and can save everything in git which are requirements for me.

Typora is a great markdown editor for something like this.

What could help is if you take pictures of the relevant logbook pages with your smartphone, and then commit them into your git repo, along with your code.

Free tip: commit often and write proper commit messages.

If you change organisation and were working on a closed source project, you lose everything. You need to keep your notes on a higher level though (i.e. not describing confidential information). Plus I believe that a commit is generally only covers the third point ("Describe the process of carrying out the method").

I keep a simple text file to log features and bugs. This approach of creating date-based files seems cool. May try it out as well.

I started doing this "naturally" :)

Reply on news.ycombinator.com