The Zen of Parallel Programming
smolnero.com
smolnero.com
I'm genuinely considering that this is just a foreword and I missed the clickthrough link to the meat of the article.
The "about" section of the website is equally vacuous:
SmolNero is a philosophical startup rooted
in first-principles thinking.
We explore how humans relate to technology—
how we speak to it, depend on it, thank it,
and sometimes forget it’s there at all.
We don’t claim expertise. We see ourselves
as translators—working at the crossroads of
emotion and programming, curiosity and care
It's not often I comment on blog posts, but this is honestly one of the most meaningless blogs I've ever read (not counting those fully AI-generated SEO ones). I don't get why it ended up so high on Hacker News; it has an interesting title I guess, but pretty much no substance.
The article makes a lot more sense if you've practiced Zen.
Think of it as someone's way of making sense of Zen teachings by related them to something they already understand.
It is just some daydreaming verbiage trying to connect totally disparate domains in a fanciful manner; nonsense and drivel.
As I continue reading An Introduction to Parallel Programming, I cannot help but notice a connection between communication among processors, communication among human beings, and communication within the individual self.
I say ... Whut?
"Everything we've looked at so far is a technical problem. Compared to organizational problems, technical problems are straightforward. Distributed systems are considered hard because real systems might drop something like 0.1% of messages, corrupt an even smaller percentage of messages, and see latencies in the microsecond to millisecond range. When I talk to higher-ups and compare what they think they're saying to what my coworkers think they're saying, I find that the rate of lost messages is well over 50%, every message gets corrupted, and latency can be months or years"
I've been on projects where everybody needs to know everything, and on projects where many groups can be insulated from others.
The more your group can work without needing a meeting with other groups, the more things can progress in parallel.
Some projects won't work well with that constraint though. And some organizations won't either.
Edit: aaaand I hadn’t read the article
But it's secondary.
The wall is build. If you can AI program, your problem is the build is too slow, too unreliable, not secure enough from a supply chain standpoint.
That is where one competent senior hacker tops out today. The agents are yielding to a CI that gets 35% per-vCPU occupancy.
The wall right now is skill or build, depending on your skill.
When you're dealing with multiple platforms, or hardware accelerators, or mostly all of economically relevant shit in the AI era you don't get a small, clean, fast build.
Fable can't print a Tauri faux-native app without dragging in half of LLVM.
If your build is bloated and slow, it would seem prudent to go and fix that, possibly converting a codebase from some other language or build system to something that can be built and tested very quickly. Slow builds and slow unit tests are a choice.
If you're not running an elite `bazel` or `buck2` RBE with `nativelink`? You're not even playing.
Having a bloated, barely-maintainable codebase and a build that takes hours and needs 100GB of storage is not a badge of honour.
In my experience with more than a couple dozen software projects, the ideal team size to successfully deliver a non-trivial effort is between five to nine people.
One subject matter expert visionary
One technical architect visionary
One or two technical masters
One or two UI/UX masters
One or two padawans capable of becoming a master
To the people wondering the meaning of the article, it's I think this.
The mythical man month is about communication between people and the amount of work to be productive in a project. It's about people colliding with programming.
It has nothing to do with a vague statement that "eventually" you have diminishing returns on processors (of course there is some limit even though different tasks could be many orders of magnitude apart).
For Brooks the communication between people starts to dominate the time spent the more people you add, so it will eventually become the bottleneck.
For me it's an interesting parallel. Feel free to come up with your own conclusions.
It's completely obvious and the "eventually" is where anything interesting happens. This comes from the early days of computers which is why something trivial became a supposed insight.
For Brooks the communication between people starts to dominate the time spent the more people you add, so it will eventually become the bottleneck.
Not just the communication between people while working in general but the communication to get someone involved in the project weighing down the people who would be making progress.
Feel free to come up with your own conclusions.
I did come up with own conclusion, the only similarity is the word bottleneck.
One is basically 1 + 1 = 2 and the other is a warning of a non obvious situation that comes from lots of experience and effort.
One is is good information from lots of experience, one is "the parts that don't use more cpus don't speed up with more cpus".
Unhelpful nonsense about running software vs battle won experience about making software.
I’ve enjoyed reading the comments here and I think there’s truth in how the technical problem is divided and teams are arranged. The idea of frequency of features (or builds) being a reflection of our division of the problem, is interesting. It made me think about our teams trying to ship releases and the problems arising, but zen and parallelism don’t give any hints. It’s just about effort to organise better, like it always was
The challenge for these “Zen and my job” types of work is that much of Zen practice is (deliberately) a bit of an inkblot test. If there’s any real Big Truth to it, it’s that it’s all one big muddle that everyone has to muddle through for themself. Unlike in many Western religious traditions, there is no presumption of univocality - even within the canonical literature, every opinion is understood to be the author’s alone and represents their own personal attempts at muddling things through. It’s just that some people and works are regarded as being particularly worth a read.
But that’s never explicitly stated. Which can be particularly confusing to people who grew up in religious traditions that do presume univocality and a greater degree of spiritual authority. Hence these works by people who see one particular bit that clicks for them and conclude, “Ah, this is what Zen is really about, I must go tell others!” without realizing that perhaps they just stumbled across an inkblot that they found to be particularly evocative.
I like the part about synchronization and honesty. And I'm certain every time I get annoyed that someone is not as open with me as I wish they were, there is a part in there that myself contributed to that.
To people who say "You don't need scalability" I say "You also don't need unscalability..."
"Zen Mind, Beginner's Mind" makes more sense in a group setting. I read extracts of it with a Zen priest in a meditation group.
A good example is game theory and game semantics. The term "actor" in this context are soo abstract that we can pretty much attempt to implement them anywhere we see fit and that I think is the beauty.
She proved that he made a trip to a town where the monk lived after his return, and the library contained a copy of his reports. She could not prove more than opportunity but she strongly suspects that Hume not only read the report but may have met with the monk as well.