The case for building something wonderful alone
derw.substack.com
derw.substack.com
This is the #1 advantage. It's huge.
I have a mind that can hold a great amount of detail, pivot, and reshape architectures; all, without writing down a thing. I can do big things, very quickly, and make adjustments on the fly. It's pretty cool.
If even one other person becomes involved (at the technical level), the whole shooting match comes to a screeching halt, as the overhead introduced by team communication is gigantic.
I have learned to break down team projects into units that can be done by engineers, working alone. This works best, when the engineers are experienced, and are very much "on board" with the development goals and heuristics. It doesn't work so well with inexperienced engineers, or ones that have not been involved previously (new hires).
That said, big things pretty much require a team. That means that the overhead is inevitable, and it is up to the managers to create a system that maximizes engineer agency, and reduces team overhead.
I highly encourage you to use a project issue tracking system even for yourself. Describe the issues before doing them. Also describe the current state of the system in a wiki. Keep it up-to-date. Then you can attract people to make changes here and there and take ownership of small parts of your system.
The main principle is that the producer should spend 3x as much time on refining and presenting what they have done, because that will save each consumer a little bit of time and has compounding effects.
I’ve been at this for a long time, and regularly have to revisit my codebases (I have quite a few). I also tend to use many of the packages that I publish, in my own work (dog food). I write code that I want to see, six months down the road, and my bar is fairly high.
I am very familiar with various devops infrastructures, having maintained many systems that integrated things like Jenkins, JIRA, Confluence, Perforce, BitBucket, Bamboo, etc.
These are pretty much required, when working as a team, but they introduce some significant overhead, and not just in the work needed to establish and maintain them. They also introduce overhead into the workflow of each user of the system.
For my personal feature tracking, I use a PostIt pad, on my desk. I have a glass-top desk.
As I plan a feature or bug fix, I write it as a task on a PostIt, and stick it on the left side of my desk.
As I work on it, and consider the task complete, I move it to the right side of my desk.
Once I have tested, confirmed, and checked in the fix, I drag the note to the trash (under the right side of the desk). I use the commit comments to codify any historical notes. I will, sometimes use a GitHub Issue to codify something for historical stuff; especially if it's something that a "non-me" stakeholder is interested in, but the PostIt for that issue is what I use. I think the largest number of PostIts that I've had, over the last couple of years, has been 4.
This only works for two main reasons:
1) I’m really experienced, so I’m pretty good at estimating and wargaming the tasks; and
2) I run in what I call “constant beta.” That means that I don’t move on to a new task, until I am satisfied that all bugs have been fixed; even the small ones. Incomplete implementation is not usually considered to be a “bug,” but it depends on the current context. Bugs don't usually last long enough for me to write them down. I'm fairly efficient at problem-solving.
#2 is a biggie. My apps are generally at “ship Quality,” almost from the start, although incomplete.
This allows tremendous iteration, especially in receiving feedback from non-tech stakeholders (I can start using TestFlight to share releases, from the very earliest stages of a project).
It also prevents those disastrous “Oops, we didn’t think of that in our project planning” issues, and also pretty much guarantees a smooth, surprise-free final testing phase.
This is especially useful, when you have a small, distracted team, like many volunteer and nonprofit orgs have.
But this is what WFM. YMMV.
You can see for yourself, how I work: https://github.com/ChrisMarshallNY#browse-away
https://github.com/Qbix/Platform
It takes care of everything from MVC to WebRTC to PaymentRequests to Web3 to Videoconferencing, and soon avatars and metaverse etc. You can literally build almost anything with it, quickly, and get it to MVP stage.
I architected it all. I had to hire developers to help me with it, and it took them years to get up to speed. My biggest source of loss and inefficiency in business was worrying that one of them would leave, after many years. Thus I constantly paid them their salaries and slightly increased them. As you can imagine, this was tough to do as an individual, so we took on projects, investors, etc.
Now, I am in full control of my two companies, and finally taking on VC and crowdfunding: https://wefunder.com/Qbix
Yes, the capital infusions can really take the company to the next level. But I can tell you that, had I been documenting (and making everyone else document) every component from day 1, we could have done a lot more, a lot more efficiently, and attracted a lot more people to our projects on GitHub. (Not that we tried, but all of these things compound and every step in the "conversion process" of a new developer would be more likely to convert). The following have helped a lot:
https://community.intercoin.org and https://community.qbix.com
https://youtube.com/c/intercoin and https://youtube.com/c/qbixplatform
This is just like when you make an app and make the UX good enough that users can actually onboard and invite friends.
Kudos!
I don't believe that I'm unique, in any way, and am not at all interested in getting competitive. Most of the work I do, is focused on NPOs and volunteer stuff. I'm not interested in making money.
> had I been documenting (and making everyone else document) every component from day 1
You ... may want to check out some of my codebases.
The code is heavily documented, and each repo has a GH Pages environment that is generated, using Jazzy.
These are all auto-generated: https://open-source-docs.riftvalleysoftware.com
The README has a bunch of stuff, so the repo is also pretty well-documented, but Jazzy actually creates an API documentation.
I'm looking forward to Apple maturing DocC. I think it still has a bit of a ways to go, and I'd like to see GitHub support a .docc file as an environment.
This is my documentation philosophy (bit of a long read. Sorry): https://littlegreenviper.com/miscellany/leaving-a-legacy/
I’m sure you’ve done some amazing work over the years and continue to do so. At the same time, these are not, in my mind, a showcase of “big” or ambitious applications.
Have a great day!
This isn't a principle, it's a specific guideline that assumes that time spent refining and presenting now will have at least equal returns later and that 3x the initial production time is the point where we hit diminishing returns. This may apply in many situations but it's not universally true.
The principle you're getting at would be better scoped as "think about the circumstances in which this will be consumed and make sure to take that into account now". If OP has determined that they are the only future consumer and if OP understands themselves well enough to guess at their future needs, they're entitled to spend the minimum effort necessary to meet those needs.
If it's a side project, working alone is a great escape from design by committee, sprints and other pressures.
I'd add to this that you can also build something wonderful for you alone. No need to please others, no need to scale. Just build it for you.
This article was almost revelatory to me: https://www.robinsloan.com/notes/home-cooked-app/
Our actual world isn’t totally broken. I do not take for granted, not for one millisecond, the open source components and sample code that made this project possible. In the 21st century, as long as you’re operating within the bounds of the state of the art, programming can feel delightfully Lego-like. All you have to do is rake your fingers through the bin.I just started literally this and it is so liberating. I got so amazed by the whole Stable Diffusion movement that I started building a UI for making a more efficient workflow when it comes to image synthesis for myself. I'm building it 100% for myself, but I also sell the UI on Patreon for others to grab if they want it, and surprisingly, others like it too, even if I'm mainly building it for myself!
But the change of pace from "We need this ASAP, drop everything else" to "Hm, this feature sounds risky but fun to implement, lets give it a try" is so nice that I'm basically getting a high from working on the project.
Decisions that might have taken weeks or months can take me less than a minute. It's incredible. And I take as much time as I need to delve deeply into researching before I move forward. For me it's not about getting done fast, it's just about doing it correctly.
I love it.
I'll go to museums by myself, write down my feelings about what I see, and resolve to never tell anyone about those feelings or that I even went. A few years ago I went through an intensive powerlifting program for 5 months and decided to never tell anyone about it.
I find a kind of power in having private things. I think some of it is owed to my extremely connected nature – I tend to overshare, and like a chameleon I adapt stuff to what people want to hear and see.
I can see some of that sentiment reflected in this post too.
> I’m forcing myself to not read the art descriptions at MoMA. I want to feel these things without intervention or a priori knowledge.
> I will not tell anyone about this
I feel OK sharing these examples on HN now, since they're all years old at this point.
I have the same feeling towards museums. My dad would drag us through museums as the default family trip when I was young. Him being the type to read each and every description, I made a private vow to avoid subjecting others to that kind of boredom.
Now I almost always go to museums alone so that I don't feel anxious about my pace relative to someone else. Almost like reading a book, I am free to study or skim as I like and focus on the things in front of me. Instead of the object of joint attention mediating communication between you and whoever else you're with, experiencing these things alone fosters communication between you and the object itself.
At some point others are added (other people support the proposal, other people contribute to the design or other people join the building effort), and that vision will not be perfectly shared by everyone.
Sure, they may all be heading towards the same general idea, but the specific destination will be different to each member of the group.
The case for building something alone is the default case. Any change from the default case needs a motivation, such as:
1. The intended final product will be too large for a single person to build (for example, a single person can build a garage, but they won't be able to finish a skyscraper in their lifetime).
2. The intended final product has an ETA that is past the time when it is needed (for example, a single person can build a small house in a few years, but that is moot if they need that house in a few months (baby on the way, getting married, getting thrown out, etc))
3. The final product needs expertise from several different industries (for example, building a shed needs only building and a rudimentary engineering expertise, while building a bridge requires sign-off from multiple experts in different disciplines).
Now, when building a programming language, it's clear that none of the above are required: a single person can build it, can do so relatively quickly, and requires no other expertise than some undergrad CS (see Javascript).
To make it popular and useful, though, requires a ton of libraries, which is where the group/team approach is needed. A single person can probably not produce a standard library that (for example) is as comprehensive as that shipped with Java or C#.
Therefore, the author is on the right track here, so far. If, at some point, the author wants to add features to his language that makes it competitively useful with other languages in that domain, he will have to start accepting contributions.
The early days are glorious though. After working 10 years at a BigCorp... when I started doing some personal projects in earnest during my time off it was pure liberation. I wrote more code in a month than I had cumulatively in several years prior. Building consensus with yourself is... easy.
I know this sounds like an ad, but it's just good.
No need to get consensus for large refactors.
experimenting with new simulation algorithms? No need to get buy in.
Spend 2 days on testing new neural topology? Sure! Why not?!
Let’s play with WebGL today, fun!
I saw a news article on the problems in forever chemicals - ok let’s pull the database, import it, and run some simulations.
Total creative freedom, working alone is great
But there's also a couple of downsides that I've encountered while solo-developing my canvas library:
- Side tracking. It is really, really easy to get side-tracked into adding stuff to the library which doesn't need to be part of the library. For instance, I love the work I've done to add a coherent graphical filters system to the library - but it has also added a lot of lines of code. Similarly with the particle physics system: the results are very satisfying, but is it necessary?
- Loneliness. While I love working on my library, the lack of a surrounding community of contributors and users can (occasionally) leave me questioning why I'm doing this work, who I'm doing it for. At the end of the day, if others don't find my library useful, then maybe I should have spent that time contributing to someone else's vision/project - what if my work is just me being selfish?
That is an awesome goal. I could use it right now! It would be great if features like the Or type A | B from TypeScript and value types could be preserved though. But that will be difficult to do.
There are projects like Reason and ReScript that do something similar with JavaScript as a target, but these projects don't seem to be mature / reliable enough to base a large project on.
But if you are going for it, that is great. Here are some features I would love for a language like derw. This is just meant to be a source of ideas you may or may not have thought of already, and obviously, your goals probably differ from mine:
1. Optimised arithmetic for 32-bit integer types int and uint.
2. Tagged unions based on uint, compiling to "const enum".
3. Provide an out-of-the box ! operator that actually throws an exception if the value is null or undefined.
4. Support for both mutable and immutable data structures out of the box.
5. Support for building reliable APIs, that cannot be breached even at runtime through some meddling JavaScript code. Currently I use Object.freeze all over the place, just to make sure that people cannot change randomly a class definition from right under me, etc. For example I have a decorator @fixed that is defined as
export function freeze<V>(x : V) : V {
if (typeof x === "function") {
Object.freeze(x.prototype);
Object.freeze(x);
} else {
Object.freeze(x);
}
return x;
}
freeze(freeze);
export function fixed<T extends { new (...args: any[]): object }>(target: T): T {
return freeze(class Final extends target {
constructor(...args: any[]) {
if (new.target !== Final) {
throw new Error('Cannot inherit from immutable class');
}
super(...args);
freeze(this);
}
});
}
freeze(fixed);The ReScript creators have pretty good experience I believe, building their own compiler based on Reason and OCaml.
I'm good friends with his sister :)
I like to process ideas externally -- talking through them and getting better iterations through feedback.
But I do not enjoy coding with folks that are not interested in alignment and bringing others with them. That is a rare breed so I have decided to go it alone to start my next project.
If I have any chance of growing it to a team I will jump at it because I much prefer building teams over tech at this stage in my career.
I am signed up for YC's startup school with my kickoff in October.
Explicitly not keeping much in the way of plans other than enjoying my time exploring my own things and spending time with my daughter with no external work stress.
It is going to be a delightful holiday season.
I have to admit that this really rings true for me, and it's why my "fun" projects are usually solo affairs.
I have had a few friends help out with some pieces, but 99%+ of the work I have done alone. I also have some people who I can discuss the architecture with but I don't need anyone's permission to make a radical change.
The project is so big that I can't possibly do it all by myself and I have had to learn quite a bit outside my area of expertise (file systems and databases). Even though I could have done it faster with a big team, it is liberating to have full control over a project.
Is it correct to assume that the core innovation here that you are parallelizing your queries? If so, have you tried benchmarking with parallel queries enabled in Postgres (eg., one test would be to increase the number of workers allowed per query)
https://www.postgresql.org/docs/current/how-parallel-query-w...
When I performed the benchmarks against Postgres, I assumed that parallelization was on by default. I certainly saw output when I performed the EXPLAIN ANALYZE operations that indicated it was working (e.g. 'Workers Planned', 'Workers Launched' and 'Parallel Bitmap Heap Scan...'). I haven't worked extensively with Postgres since version 9.5 so I haven't kept up on all the configuration settings.
After getting this message, I went into the Postgresql.conf file and bumped up the values for max_worker_processes and max_parallel_workers. This seemed to make some of the queries a bit faster, but I have not re-run the benchmarks in the video.
The amount of things I've learned by going it alone on past and present projects...
Working alone meant I could do things at my own pace, be the 'benevolent dictator' to myself, and to style a look & feel for my Thing(s) the way I want it to be.
As an example, my ongoing project which is a 3rd party app for Explorers in Elite: Dangerous: https://captainslog.scarygliders.net/captains-log-2/
Python, PySide, sqlalchemy, etc. Even a bit of HTML, javascript and CSS for things like the animated logo in the About window which is a QWebView, and some other web views which I managed to blend in with the Qt widgets. There was so much to learn and so much has been learned, and even that's an ongoing process.
I could open source it, and have pondered exactly that over the years, but I just don't want the extra overhead of "herding cats" - props to those who can and do such though. In fact I have open sourced a couple of things but - perhaps thankfully - no one seemed that interested in participating.
Something which you have complete ownership of from conception to completion.