The software engineering notebook
winterflower.github.io
winterflower.github.io
It's indispensable in meetings because I both take notes about the goals of meetings beforehand, take note of any important parts during them, and am able to succinctly review them afterwards as they relate to the original goals, and easily keep track of any action items.
I manage EVERYTHING this way now-- what I am currently working on, notes about what I've done in a day, notes about what I've done on the weekend, notes about books I want to read, movies I want to watch, have watched, fitness routines. It goes on.
At first it was a small but useful knowledge base, and a good plan for how to keep track of things better. Now it's a larger, and even more useful knowledge base, where due to review and organization the important concepts in it have mostly made its way into my long term memory. And the process of taking and using the notes has become a smooth and well-honed process. I keep the notes in dropbox so I have easy access to them on mobile also.
In short: I treat my personal knowledge base as if it were a coding project: using well organized text files. I am constantly adding to them and maintaining them. It is working very well for me.
One thing that wasn't working for me was storing TODO style things and "X I want to watch/read/learn/play/do" items in my notes. Eventually I found Trello and it seems a good fit for that style of task (for me)
Everything else is stored in my notes though, or sometimes in paper-based notebooks that I take pictures of occasionally for archiving.
Still need to get better at things I've done at the weekend/daily bullet points etc, I often forget or am too lazy
The importance of the 'reset' has helped me a lot too, and calling a lot of recorded things 'baggage' is very true. If everything I had recorded was wiped would I be disappointed? Definitely! But life would go on. :) I'd re-discover things, my tastes and personality would change so those recorded things aren't relevant, etc.
I have system of using "top" tags to define my working view for a project. Say I'm working on project Foo. At any one time I'll have about half a dozen notes tagged "foo-top." I can take any note from my past and bring it front and center by tagging it "foo-top" and then delete the tag when I want the note to disappear again. Say, for example, I want to see if CouchDB will be useful for Foo. I used CouchDB for a personal project several years ago, so I can search for my note of CouchDB concepts and commands, tag it "foo-top", and then remove the tag if I decide it won't be useful. It's a beautiful way of carrying around a lot of baggage without feeling the weight. It's nice that if I want, I can write down notes and thoughts about something without worrying that I'll clutter up my journal and make it unreadable, or worrying that I'm mixing up the wheat and the chaff. It's not like I spend a bunch of time rummaging through a bunch of junk to find what I want, forcing me to think carefully about what I record. I can record anything that seems like it might be useful later.
A journal would be nice to have, too, though. One note in each project serves as a journal of sorts, in the form of a quick summary of what I've done each day and a quick summary of what I expect to accomplish next. (For work, I find it more convenient to keep a single note for this instead of breaking it out by project.) Evernote isn't great for keeping chronological notes. I only use it because I haven't seen an alternative that is compelling enough to make up for the convenience of using just one program.
One thing I really wish for is a program that could automatically present me with a diary of sorts based on my note-taking. Versioning would be nice, too, so I could see how my notes evolve over time.
Is there not an API in Evernote to query the notes? Seems like getting the last few weeks’ worth, sorting, and dumping to html or pdf would go pretty far for your diary use case.
My notes track knowledge, they are referenced as-needed so I store everything because there's low/no cost. I've also found that writing things down improves my recollection, so there's value even if I never reference the note again.
My calendar / omnifocus track future actions, I minimize these and keep them very well curated - no "todo", only "do". Externalizing some cognition takes time in exchange for efficiency... less overhead from context switching or remembering unrelated items.
The tradeoff is loosely analogous to scheduling in computing - inefficient for an isolated single thread of work, but beneficial when coordinating lots of concurrent work.
I'm curious about what this means. What are "do" notes (as distinct from "todo" ones). If it means that don't calendar, and you just do that task immediately, how you plan and (more importantly) remember future tasks?
Grep any large code repository for "TODO" comments.
However, I think code and life are slightly different contexts here. I get what you mean about separating aspirations from near-term actual intents but I generally use the word "todo" to refer exclusively to the latter (which still doesn't help a whole lot with ensuring execution as the list still grows quite large).
I keep the two separate: good notes describing the state of the world, and a calendar to schedule future actions.
When it comes to software, that means all code must have good documentation and known flaws are scheduled to be fixed. If it's not worth scheduling, it's not worth changing.
My personal life follows the same pattern, although much less formal... there just isn't enough time for every possibility, so I always work backwards from my calendar.
I've mentored enough colleagues that I've come across those who take extremely detailed notes, and are successful, and a fair number who do this and don't find success in their work. When I try to find ways to get them to question their approach they are often resistant to changing because the have an ingrained belief that detailed note-taking is of fundamental importance.
Which makes me think there is something not so obvious that accompanies effective note taking.
Taking notes about routine everyday stuff is boring and worse than useless, because the notes are full of duplicated information that makes searching slower. I imagine the people who take notes for it's own sake are basically listing everything that they do, whether they gained any new insight or not.
I spent my entire education and most of my software career taking few notes, and they did the job well enough. Despite my shift to taking a lot more for various reasons, I agree that there is no moral or strategic value to taking nice notes just for the sake of having done so.
I read my notes every now and then. But what the notebook does mostly is make me focus and concentrate on my goals and remind me what's important. As the parent said, a notebook enhances your long term memory. You re-iterate lessons learned/forgotten. It's not a bookmarking tool.
As part of treating my notes as a code project, I even keep them version controlled, which leaves me a little more free to just delete things or "refactor" stuff. A lot of the value I get is going back through things and synthesizing them into clearer and more useful notes. TODOs in particular are dangerous -- I try to treat files of that nature as extensions of my long term memory, rather than a curriculum which I feel obligated to follow.
That being said, there is an element of this project where I feel like a am building a piece of "my life's work". I've built a number of software projects, and part of building them over time and organizing them leaves them in a state where I am proud to have built them -- I can point out a piece of the system, and understand why it works that way, and remember the choices that led to its current state. At one point, I realized that I had no such project for my life, and it was a little weird that I had so clearly documented software systems that were hardly used, whereas my life and action and ideas and knowledge were, for the most part, just in my head. So in a way it's more than the practicality of having notes, although in the practice of it I have found it to be practical. It's a way of adding meaning to the course of my life -- now the ten minutes I spent reading an article can also become a gift to my future self due to the one choice quote I grab from it.
Since 2013 I started creating a text file almost every day in a directory structure of '../daily/2017/17-10-14.txt'.
I use it as a journal (I put the current time and wrote whatever it is down), a scratchpad, a temporary todo, habit tracker, etc. It's been super useful for me. I've thought about formalising it, using tags, etc. but have found myself getting along fine.
I used to use Day One (macOS/iOS) religiously but have found that this daily txt log mostly sufficient. What I would like is to expand my macros from getting the current time to getting my current location and weather.
For work, I have a 'log.txt' in which I have a section for each day and copy-forward the previous day's section. While we have our own company tracking tools, etc., this is purely for me and has helped me keep better track of everything I need/want to do without letting anything slip out.
I've found that doing each of these things takes up very little time too—partly due to not formalising it or making it fancy. I've thought about using a vimwiki which I'm a big fan of, and gave it a go once, but I stopped. It was no fault of vimwiki itself, but so far have found the text files sufficient.
For my phone (iPhone), I use https://ia.net/writer/, and I keep all these text files in Dropbox.
Another reason why I've stuck to text files is because of the fatigue of different apps and services disappearing due to whatever reasons.
When I need to find something old I use fgrep on that folder. Works like a charm.
It's plain text, no defined structure. I dont't use it for TODOs as that info would get easily lost.
I use it almost everyday, specially when something different happen: how to release an old piece of software that current setup dont support anymore or how to connect to a service that I only use sporadically.
I tried some tools and apps for this, even tried to track that for a few days in Trello. eventually I gave up and went back to the plain txt file.
I.e., how often do you actually read the things you write down?
PS: Have you ever thought about using a "lifelogging camera" instead? :)
BUILD QUALITY RELATIONSHIPS - improve existing relationships, and seek out new good ones
MAINTAIN NEUTRALITY - approach things with a beginners mind
PRACTICE SELF-COMPASSION - admit pain, take kind and mindful action ")
Some files are just used a lot for both reading and writing. These include a todo.txt, event_planning.txt for planning logistics of whatever fun getaway or event I'm doing next, a shopping_queue.txt (reduce impulse buys by requiring I put things on that list for some amount of time).
I find that for taking notes on books or presentations, I will write them once, and then refer back to them infrequently -- maybe once a year. A bigger advantage is gained because it makes it easier to stop reading/watching things, because then I can always pick it up again and review the notes associated with it, so I can have a lot of educational threads going at the same time without fear of losing the context.
For journaling, I tend to review my journal and list of activities as a whole a couple of times a year, and typically will review the last week every week or so.
Much of this happens naturally due to the fact that the project is open -- it's often easier and more interesting to review my past work week or pick up where I left off on a book (stored as a pdf within the notes) than it is to jump onto facebook or whatever.
The organization of notes in an accessible manner is a challenge for me - perhaps I'm doing it wromg.
Otherwise, since these notes in the context of a project, I can treat it the same way I treat code. So a mix of multi-file search, and directories that correspond to the knowledge in them (e.g. "finances/investing.txt" or "journal/happiness_project.txt", the latter containing any work I do on http://diurnal.st/2013/05/27/training-your-brain-to-be-and-s... -- which I don't do everyday by any means, but that's an example of the kind of thing that exists in my notes)
My system is quite simple. I have created a github repo called journal and an alias which opens vim to a markdown file for today's date. If I leave the file and come back later vim will drop me in at the end of the journal. I intend to add some vimscript in the future to automatically add a timestamp when I reopen the file but haven't gotten there yet.
This system has been quite helpful to me as it is resilient to data loss as I can push to multiple backup systems. Easily searchable (grep). And can support prettier documents if I want to open my Markdown formatted journals in tools like Macdown.
TL;DR I use vim to manage this. See alias below.
alias journal='vim + "/Users/username/journal/$(date +%Y)/$(date +%Y%m%d).md"'
alias journal='vim + "/Users/username/journal/$(date +%Y)/$(date +%Y%m%d).md" -c "execute \"normal! Go$(date +%T)\<CR>========\<CR>\" | startinsert "'alias journal='mkdir -p /Users/insert_your_username_here_on_mac_on_linux_change_Users_to_home_on_windows_good_luck/journal/$(date +%Y)/; vim + "/Users/insert_your_username_here_on_mac_on_linux_change_Users_to_home_on_windows_good_luck/journal/$(date +%Y)/$(date +%Y-%m-%d).md" -c "execute \"normal! Go$(date +%r)\<CR>========\<CR>\" | startinsert "'
[1] http://www.webpages.uidaho.edu/mindworks/Capstone%20Design/P...
95% of them go into what I call my "dev diary". Each day gets a new file. Any questions, answers, thoughts, etc. while programming or designing go in there. For searchability, I also add any reference Jira/Trello/etc ids.
Beyond that, I have cheatsheets and checklists. Cheatsheets are short, useful things (e.g. commands, undocumented gotchas, etc.) for a given piece of tech. E.g. I have a git.txt, mysql.txt, etc.
Checklists are things I should go over whenever I do X. Am I about to commit something? Well, there's a list of things I tend to forget to do. So I have a checklist for this.
Each project also gets a file (or folder if it grows large enough) in my notes directory.
I previously used Evernote. Then Onenote because it was nicer on the surface pro. But I had to abandon Onenote because they don't have a good Linux client. I decided I was going to stop throwing my notes away and just switched to regular files.
I've found that for some reasons I simply cannot consume/read .md in source form.
After all having <h1> rendered in larger font means something, right?
Sits in a browser tab, insanely quick to use which is the most important thing.
Consider it perfect for taking notes.
I use seperate systems for todo and project management (the project management one I wrote myself)
These files have useful commands, todo lists, debugging notes, stack traces, useful links.
Don't overcomplicate things - just make a text file each day and dump everything in. Don't bother too much with structure or tidiness. If you need info from the previous day, just copy it across.
Once you've got enough stuff in your notebooks to be valuable do remember to get an automated backup going.
It's one thing to keep notes. It's another to have a social network around notes.
What's the signature for?
* no explicit saving or loading, everything automatically persisted as plain text
* infinite persistent undo / replay of text entry, for retracing steps; can double as work logging; this uses a log file alongside the plain text; this also lets text be deleted without being lost, which helps with clutter
* "most recently edited first" page order, so you can forget about those old pages
* minimal ceremony to create a new page; use the first line as the title
* quick searching and navigation in time and space, using keyboard
These days? I rely heavily on Trello for personal projects - organize things by keeping a separate Done board (list for every month) for the project for times I want to flip back through older work done. Also keep paper notebooks around for times when I want to sketch ideas out.
Additionally there's a 'rhizomatic frankenstack' of nvAlt+simplenote / pinboard / markdown files in project folders etc. I keep wanting to set up something where it's all searchable and ideally dream of something like federated wiki with a lot more UI polish.
[1]: term from Venkat Rao's post http://mailchi.mp/ribbonfarm/frankenstacks-and-rhizomes?e=96...
And yes, I am looking for a new job.
It has a Code Macro plugin that supports Pygments syntax highlighting. It all comes down to how you structure the wiki (I often turn to Wikipedia for help with that). I've been an Engineer for 20 years, and the info-flood doesn't really end, unless you dial back your enthusiasm. I like to document the tools and services I discover so that I know what's valuable, and what's out there, should I ever need them.
Once installed, its very easy to create new pages and edit them. Plus there are hundreds of plugins.
https://www.tbray.org/ongoing/
I keep my own blog entirely for my own benefit, and the tech section is simply things I've read recently, or things I've learned recently:
More importantly though I use my blog for keeping more polished, more elaborate versions of my more relevant notes. Sharing your thoughts with others not only benefits others but it also makes you think over and refine your original ideas.
Distilling your thoughts for the benefit of others will help you better understand the matter as well. While we teach, we learn.
Plain Text:
Pros > Simplest, works everywhere/on everything, scales well (append forever!)
Cons > Least automated, no images, no math, no code, hyperlinks depend on editor, no embedded files
WYSIWYG Doc Editor (Word, Writer, etc):
Pros > As easy as plain text, can do images/math/hyperlinks/embedded files, somewhat portable
Cons > Bad at code, not many automated features, unwieldy once file gets large
Markup language tool (Org-Mode, Wikis, etc):
Pros > Powerful, portable, good at code/links, scales well, highly configurable, highly scaleable
Cons > Have to learn the markup language, mostly bad at math, high up-front costs (e.g. needing to set up a server or learn alien keyboard shortcuts), non-wysiwyg feels less like a "quick note" and more like a professional document, image/doc embedding can be hit or miss
Note-taking app (Onenote, Evernote, etc):
Pros > Automates lots of note-taking tasks (e.g. time stamping/tagging), can do images/math*/hyperlinks/embedded files, scales... fairly well
Cons > Least portable (dependent on app-specific sync features which may change), less simple (best use of notebook features are non-obvious), still bad at code
Electronic lab notebooks (ELNs) (labfolder, benchling, etc):
Pros > Good for organizing/presenting data, embedding files, scale very well, have collaboration features
Cons > Very not-simple (most require either require server config, or you don't own your data), most are somehow still bad at math/code, frequently expensive and not portable
Task Managers / To-do lists (Trello, Asana, etc):
Pros > Very intuitive to use, can sort of embed documents/images, have collab features
Cons > Not good for longer-form notes, bad at math, not meant for note-taking, you mostly don't own your data
Others:
Jupyter/Mathematica can do WYSIWYG text, code, math, data, and images all in one document. However, Mathematica is expensive, and Jupyter is non-trivial to get up and running. These tools are primarily geared towards creating a polished document, and are less good at documenting the evolution of ideas inherent in note-taking.
Personal Diary/Journal software is mostly underwhelming, most of these seem like cut-down note-taking apps sold to the non-tech-savvy
When I tried following their instructions in the past, I was unable to install the kernels I was interested in (Haskell, F#)
Edit: My current stack, to show good faith: paper for log and tasks, Org and Git for memoranda.
For me, note-taking is something which helps me digest complex ideas. It's a valuable thing and something you can improve at with time.
Then I read the reasoning behind why others don't take notes and I stop to think. Hmm, maybe there is such a thing as too much. Maybe note-taking has a fine balance. Breaking up complexity and summarizing it with notes can be very helpful with getting something finished. Writing journals of everything you do daily probably detracts time from actually performing the tasks required.
There's some science out there on how dopamine gets released when we TALK about performing an action with desirable results. (I'm going to create this project, I'm going to workout this summer and lose weight, I'm going to read 12 books this year). Basically, you're being rewarded for not doing but just saying. If you spend too much time note-taking on a plan or project, you might be getting the dopamine fix, which might make it harder to execute.
Similar comparisons are with startups and large companies. Startups work fast and can ship products quicker, at the cost of maintainability. Large companies have to take their time documenting, grabbing requirements, assigning the team, and following internal processes. The process kills productivity, in favor of predictability and maintainability.
So after thinking about it for a while, I realized that the best way for me to take notes is just to have a chat room with myself, where I write things down as I do stuff or think about stuff.
The common pattern is that I start the day by writing something about what I want to do, how I plan to spend the day, maybe some reflections on how I feel about the project's direction... Then during the day I'll put some brief notes, including random ideas that I just want to get out of me... And then at the end of my workday I'll go and reflect what I did and the current status.
Then I also have a setup of Org mode documents. One is a general project management file for all the projects in my life (housing, volunteering, work, recipes, etc) and then there are notes files that are more like sprawling personal wikis or (more pretentiously) like very disorganized book drafts where I flesh out my thoughts in a more long term way or in depth way.
(About the self-chat, I also use that for writing things like blog posts; I make a first draft just by chatting, and then I spend hours reorganizing and editing.)
https://www.npmjs.com/package/livecomment
Samples here:
http://acpul.org/livecomment-src - livecomment js sources (it's quine, that developed using self documented code)
http://acpul.org/pool - common memory pool: active info, some processed bookmarks from HN, reddit, etc.
http://acpul.org/pica - livecomment in real project. here is collected info when i was develop gpu optimizations for popular nodejs image resizing library https://www.npmjs.com/package/pica
I still keep copious notes every workday of everything; e.g., hyper parameters for modeling, server info, URIs to useful documentation, etc.
Outside of that, a friend of mine has made his own semi-public wiki, and tied that into his email client; he can now easily link to pages from within the email, allowing him to communicate A LOT with very few new words. It's pretty neat.
Working on this has definitely helped me learn these concepts much better and deeper, plus it's easy to go back and find things when I need them. Just reading something is not a good way to absorb information. Actually having to regurgitate something in your own words makes a huge difference.
Gitbook is not the best place for all my notes. Will be using Evernote going forward.
* Leave the office once you're really sure that deployment was successful
* Have timestamps in logs
* Before making your opinion public, make sure what you think is a mistake is in fact a mistake
* When you struggle with an issue, make sure the fix isn't right in front of you
* Make sure everything's set up for remote work; you don't want to hotfix some production issue at 2 AM, only to find out you can't access Git from home
* Other people's time is worth twice yours
I used to use SideNote, but that was abandoned. I'm currently using SideWriter, but it's a little flaky.
I hope to start learning swift soon so I can develop my own with some extra features, mainly markdown and either tabs/multiple hot zones.
My memory is horrible, so I need to write down what I did/thought before it vanishes. Mostly comes in handy when I have to justify to HR why they need to give me a raise.
I started with old wikis and then a Russian PIM that I can't remember and then Google Notebook. Evernote was always a hit o miss the sync never was good for me.
I have a $700 computer in my pocket and the best way to write a grocery list is on a post-it found in the kitchen drawer.
I think it comes down to the human interface of a touch screen being slow, frustrating, and generally inefficient.
O horror!
Useful for taking notes wither portrait (in meetings) or landscape (in front of a computer). Also graph paper obviously helps with sketches and dividing up the page on the fly.
Sounds stupid but I've yet to find a better tool.
I'll use Org-Mode if things get more complicated, but nothing beats plain text.
I do use notes.app, evernote for complex writing stuff though.
Edit: I take that back, Boostnote looks pretty good!
There seems to be some UI slowdown on my machine too, whereas Quiver doesn't seem to suffer from that at all, and I have like 200+ note files in my Quiver repository!
I must give Boostnote a try. On the face of it, I would think an engineering notebook, to be used daily for large amounts of stuff, would be a paradigmatic case for a native app like Quiver rather than electron. But then there's Atom & vscode so maybe not? And I'm a bit uncomfortable with Boostnote's copyright transfer for pull requests (https://github.com/BoostIO/Boostnote/blob/master/contributin...). But then again, I pay little attention to copyright matters so perhaps just haven't noticed if this is common in open source. And it's clearly nevertheless more open than Quiver.
But this setup is somehow clunky because there is no tracking if a ceratain topic/headline already exists, linking is not possible, etC
I recently moved off Basket and after an extensive search, found QOwnNotes seemed to fit my brain the best.
No affiliation, just like and use it :)
And physical notebook for other things.
Any article I like on the web which I might want to come back to in future, Evernote makes it easy to clip web pages as is.
What's the point? After some time, that would the list of things you forgot... :)