Coping strategies for the serial project hoarder
simonwillison.net
simonwillison.net
If I want to do some feature work on the thought I create a GitHub issue and write implementation notes, I did this on samsquire/hash-db for document storage.
I write English descriptions of the mental model then implement.
Some people say code is the documentation. This is unfortunate and means that thinking ends when the code is decommissioned its lineage ends and cross pollination never goes on.
The lifespan of an English description on Wikipedia of an algorithm or project description outlives its implementations.
Please document your mental model of the problem you're solving.
As an analogy the documentation and thinking behind the implementation of Windows 95 (and it's user interface design thinking and guidelines) is outlived by its documentation and screenshots.
University and research publications whitepapers outlive the code written during research and potentially many implementations.
McConnell has a really good quote on this from Code Complete:
The information contained in a program is denser than the information contained in most books. Whereas you might read and understand a page of a book in a minute or two, most programmers can’t read and understand a naked program listing at anything close to that rate. A program should give more organizational clues than a book, not fewer.
Saying that a reader understands a page or two of most books in a minute or so is like "understanding" the rules of Chess but being unable to actually win a game.
What differs between books and programs is the consequences of getting that sense of understanding wrong. When reading a book, the results of misunderstanding or skimming are rarely serious (outside the realm of technical manuals, textbooks, and holy books) whereas the programmer and user usually immediately notice that something is very wrong if the programmer took a wrong turn in their understanding.
Academics paid to "understand" non-technical books face so little consequences for misunderstanding that no one even agrees what understanding is. James Joyce's Ulysses is a crypto-ur-fascist work prefiguring the ascension and demise of Nationalist Irish America as found in the correspondences between Buck Mulligan and Donald Trump? Well, I certainly can't tell you that's wrong. Half of these terms would require volumes themselves to meaningfully define to an extent that a conclusion could even approach falsification.
This failure to understand misunderstanding is also pervasive in the majority of legal systems. They try to define terms and precedents in volumes and still have dogmatic disagreements about meaning and understanding.
Try doing this in a codebase and see how far you can get. "As you see, this comment in the Linux kernel was written by Linus Torvalds. Torvalds had many documented conflicts as a result of his frequently acerbic personality. We can date this comment to after his foray into improving his interpersonal skills. We can infer from the use of language found in the manuals of the University of Oregon's Anger Management school that Torvalds is emotionally disturbed yet diplomatically declining to accede to the Rust development team's demands for special privileges. Seeing that Torvalds, the creator of Linux, wanted to originally decline these changes, I have put forth a commit to undo all commits initiated by anyone associated with the Rust development team."
On the other hand, if programmers were to instantiate legal and semiotic assertions in Prolog from first principles, suddenly there's a legibility and granularity that makes the entire "meaning" and "understanding" game pointless because we can directly see and talk about where the logical assertions fail to correspond or contradict.
I enjoy using Quora.
I generate random questions based on permutations of templates a bit similar to a madlib generator but for questions.
https://github.com/samsquire/human-query-engine
I had another project that could collect salary data and stock market portfolio's based on statements you add to the system. it was a personal data collection syste. that was programmed to produce useful functions based on what you tell the system in the form of templated statements.
The closest software that existef at one point in time is Mozilla Ubiquity which I loved. It was a twitter CLI for Firefox.
The idea was meant to be used to give targeted advice and judgements or evaluations to people. It's a bit similar to a choose your own adventure book but with advice or a problem flow chart.
This attempt is here https://github.com/samsquire/living-documents And here https://github.com/samsquire/fact-collector
This one uses prolog to collect statements then shows statistics of your collection.
Think of it similar to Siri in reverse. Rather than asking Siri for an answer for a question, you are asked a question and the answer is used to generate something useful. The problem then becomes thinking of useful questions that you can do automation over
Simply having a twitter style tweet that allows you to say "I am at London", "I am eating this" then calculate calories or nearby trains home or travel information.
My other attempts are here This one calculates train journeys using neo4j It parsed plaintext statements for a "I want to travel between <station> and <station>"
> If you build something that people can sign into, that’s not a side-project, it’s an unpaid job. It’s a very big responsibility, avoid at all costs!
Interesting for sure. But what about those of us that hoard side products?
Good insights. Both great steps to being able to re-learn & experiment (or maintain/fix) quickly.
Last year I started to break up Common Lisp code monoliths for personal projects into small libraries that are installable with Quicklisp, with applications also Quicklisp installable that are smaller by reusing libraries. I have started doing this to a lessor degree with my Python side projects and in the last week I set up a personal library system for my occasional Chez Scheme projects.
I bookmarked the author’s two templates for new Python library and command line tool projects.
Then I was working in the blockchain space and so I decided to build a lightweight quantum-resistant blockchain using my pub/sub library for peer-to-peer messaging... Then after I finished the blockchain, I decided that it would be fun to build a decentralized exchange on top of it and so I did. It's been running for a few years without issues though it's low volume but the community around it is dedicated.
Now I'm looking for new things which I could build on top of the blockchain and DEX. I'm thinking to use it as a payment system which accepts multiple cryptocurrencies interchangeably. I have a few ideas but I'm more focused on earning money nowadays so I'm hesitant to start anything new. Lol. After all that work, I only earn about $1000 per month in passive income. I'm a brute-force entrepreneur.
It's pretty easy to maintain all this. Like the author says; tests help, but even more important is to keep the number of third-party dependencies to a minimum and to avoid using overly niche programming features or features which are unlikely to be forward-compatible.
Ironically, I toned down my enthusiasm for this author's (many) projects after my initial perusing led me to something interesting[0] that didn't work, and the subsequent issues and minor (but linked to issues!) PRs I contributed went completely without response for the last few years. They're still open.
To be clear, I'm grateful for the work the author is freely providing for me and the world! And I could certainly do a better job with some of the projects I help maintain as well. He's under no obligation to respond to issues if he doesn't have time or just doesn't want to. But it does speak to how difficult it can be to maintain over a hundred projects, even if you have a system.
I've decided that learning to live with that is the price I have to pay for running with way too many projects!
I would love to watch a day of programming by someone as prolific as Simon to see what other things he does to maintain speed and keep churning out code!
Unless you care about project portability. In which case, GH issues are obviously the wrong, locked-in place to put things.
Not that there is an obvious solution to this problem. There are a few tools trying to fix this, but they're still a bit awkward. I wish Git added a somewhat-native/idiomatic way to deal with it.
I reckon git has already all the low-level features it needs to make this work, it just needs a bit of porcelain on top.
> How do we link to these issues
The same way we link to commits today? As in, we don't - not in the tool itself. The critical part is that such managers will want a nice web interface, not a command line tool. That's the porcelain I mentioned.
The counterpoint to this is that GitHub has been around for a long time and I don't think all of these free features are going anywhere. Even beyond that, there's an API you can use to get your data out and lots of tools build on that API.
https://github.com/MichaelMure/git-bug came up here recently and that looks pretty good, though I haven't tried it. I suspect the usability of a lot of these tools will lag behind GitHub Issues/Projects.
It's still an external system you don't control the fate of, run by an private company, fundamentally unaccountable, and a massive single point of failure. For some people, this is not a good thing. It's a bit like using GMail vs an actual mail client.
“Massively increase your productivity on personal projects with comprehensive documentation and automated tests”.
Is the possibly more enticing subtitle, for people who don't automatically click when they see the simonwillison.net doman, but frankly it goes beyond that to something more like "Getting Things Done for open source side projects"
This could fall under tests, but I've found that having concrete steps written down on how to build the thing helps. As there's many times something that's easy to forget that has to happen for things build correctly if I've stepped away from the project for a while.
I suppose it depends on what you're trying to do, but the thing about personal projects is that you don't have to maintain them :P
But I could of course be wrong.
But I've found the opposite to be true! My capacity for personal projects dramatically increased after I started habitually writing tests for them.
It turns out tests aren't just useful for large projects with multiple contributors: they enable horizontal scale for small projects with a single contributor too.
On projects that are well covered with tests, I find I'm less afraid to pick it back up again and start working on it, because I know there is less chance of some missing context I had in my head months ago and now is forgotten and my change will break the app.
Most of my small projects have automated deploys using ansible, not because I couldn't do the deploy manually (in most cases it's ssh to the server, git pull, run migrations, collect static, restart the server), but because with ansible I don't need to remember the manual steps.
I am sorely lacking in the documentation area tho :-) Your post prompted me to try to improve there, as I can definitely see the benefit.