Minimal Viable Programs (2014)
joearms.github.io
joearms.github.io
New features mean new sales opportunities, good for the company but not good for the user.
The problem with adding features to MVP is that when we ship more complex products like complete operating systems that are packed with programs, the complexity of the individual programs contributes to the complexity of the whole.
Not sure how to solve that particular problem though.
This made me think of a blogpost linked on Hacker News when Joe passed away: https://github.com/lukego/blog/issues/32. This part seems relevant here:
Joe wrote amazingly simple programs and he did so in a peculiar way. First he wrote down the program any old way just to get it out of his head. Then once it worked he would then immediately create a new directory program2 and write it again. He would repeat this process five or six times (program5, program6, ...) and each time he would understand the problem a little better and sense which parts of the program were essential enough to re-type. He thought this was the most natural thing in the world: of course you throw away the first few implementations, you didn't understand the problem when you wrote those!
git add program
...
... #first implementation works; time to rewrite
git add newprogram
...
... #second implementation works; bury the first
git rm program; git mv newprogram program
...
It's preferable to work in branches and rewrite and refactor incrementally whenever possible, merging the new code in as things progress at convenient milestones.
Then the history documents which functionalities in the old implementation became what files and lines in the new implementation.
That information can be very helpful when the rewrite breaks previously working functionality and customers are upset, especially if the rewrite authors aren't even around anymore.
That's the goal of this practice. Throw it all out and rewrite from scratch.
> That information can be very helpful when the rewrite breaks previously working functionality and customers are upset, especially if the rewrite authors aren't even around anymore.
I'm talking about writing non-releases. The entire structure of the code, algorithms, data structures, API / CLI must not be preserved. Anything that stays constant between non-releases is either (a) a good idea or (b) a stone unturned. Rename variables, split and join modules, violently reorder function arguments... nothing is off-limits. After I've been through a few rounds of this, the first release should be entirely free of inessential warts (and maintenance is typically quite easy at this stage).
After the first release, I'm a stickler about backwards-compatibility and my development pattern is closer to what you're suggesting.
Among folks who know me, I've got a reputation for writing clean, optimized code. It isn't because I'm good at writing clean code; it's because I'm good at throwing away ugly code and I'm very reluctant to share code that doesn't meet the bar.
Yes yes, have points, well done, but it’s a boring low effort class of comments that applies to nigh every single idea ever, and skips the interesting or important bits - in this case whether it produces better software or helps understand things - in favour of a cheap vote grab. “It’s not universally applicable” - no, nothing is.
and you don’t sound cynical you sound like you’re trying to dismiss and status grab, like someone talking about the “out here in the real world”.
Still, my point stands: I am not sure if this is really a matter of “SOME people can’t do this”, I suspect it is the other way around: “SOME people can do this... because they are working on free software, on their own personal project, on programs or systems that do not have any TTM aspect”.
And it is not just a matter of "deadlines" either: if your code is part of some bigger system you may not have the chance to reimplement things from scratch not because you don't have time, but because you could impact someone else using your library/module/API.
In other words, this approach is eminently laudable, but in my experience does not scale in most cases.
> if your code is part of some bigger system you may not have the chance to reimplement things from scratch not because you don't have time, but because you could impact someone else using your library/module/API.
In this case, if your code is used by someone else, you can't do that. The trick is not releasing your code "out there" until after you've done several rewriting passes on it. In the cases I've done that in professional context, I worked on a feature branch. I would occasionally rebase it on top of master, to stay current with rest of the product, but nobody ever looked at my feature branch until I've done a few rounds on the code, declared it's ready for integration, and issued a merge request.
I suppose this should teach me to wait until I have time enough to write a proper post, next time.
My situation isn't typical. To put it in common parlance, this is how I nucleate my 20% projects, and personal projects. Which, once released, almost never require significant maintenance because of the care and attention spent upfront.
I'm more than capable of whipping out a prototype that will do the job and incur technical debt. And I'm upfront about tradeoffs in planning, and meticulous about the rare deadline I get saddled with.
In my experience my MVPs often end up in production and that utopia of iterating in isolation is fairly short-lived.
You’d need to find a niche that values this level of quality, but if that exists, and if the theory is true, then there’s your gold.
I come to think that the whole concept of MVP/prototype became a bane to software development when the management/business side got aware of its existence. Architecture and design sessions can be skipped because we just build a prototype.
I have yet to see a prototype that did not end in production. It's "good enough software", let's move to the next feature.
When you are really lucky you can revisit your prototype a year or two later and try to improve it's design now that you got some data on its actual usage, but you have to figure out again, what the heck you actually did...
I think it is less a problem to be solved than a fundamental trade-off that comes with the scale of systems: the more components a system is going to have, the greater the cost of not minimizing complexity of individual components.
https://course.ccs.neu.edu/cs5500f14/Notes/Prototyping1/plan...
They used to... Maybe even when this was published in 2014.
I found that tutorials often had unnecessary amount of complexity to show people how to do something, like encumbering an example with classes, title setting, etc, none of which is essential to the operation of the program and which makes it difficult to understand and learn.
So, if I found a working example, I would strip the example down to its most essential until I cannot take away anymore without it ceasing to function.
Maybe I should call it minimum viable examples instead.
But I'll be damned if I don't use `ls -lah` because I am a human, not a robot. `ls -1` is also super handy. I am sure there are lots of others that folks use.
So where do you draw the line? It makes way more sense to include a few extra options in your tool rather than requiring your users to jump through extra hoops to get something _they_ find useful.
edit: On the other hand, you don't need to include the kitchen sink, either. I have seen programs with literally hundreds of long-form command line arguments. It's a nightmare to wade through.
I'd prefer that '-lah' be an external composable modifier. 'ls -la' looks like it is, I think, because it tries to create simpler common case, for the expense of that case be more specialized. Since, as somebody decided, 'ls' by default contain only some information available, one presumably needs 'ls -zz' (a made up option) which would produce all information available, and a composed to it filter which would select what is needed at the time.
If composability would be really cheap, we wouldn't need that special '-zz' option. We can even consider an option as a composition of additional operation over generic 'ls' behavior - in this case we wouldn't need the shell 'pipe' operator, and syntax of shell statements would become closer to Haskell's. For example, 'cat' could do something generic by itself, while 'cat note.txt' would be considered a composition of 'cat' and 'note.txt' which creates a specialized behavior (which sort of reminds Forth).
Also, I think these presentation/formatting flags are a consequence of the Unix shell "everything-is-text" paradigm. In Powershell, where you pass around structured data rather than text, typically you handle presentation just by piping into the appropriate cmdlet.
Oh man... that is so accurate it hurts.
Management wants 1 issue at a time/person. So they just mention the next task in person, that you are not allowed to create on the board.
Lol :-(
http://stephenbalaban.com/static-site-generation-in-50-lines...
For example tmux, if you look at a cheat sheet you might be overwhelmed, but when I teach people to use it they find it extremely simple. I just tell them ctrl-c to create a new window, and ctrl-w to switch windows. They only need 2 commands to get value, even though there are hundreds of features, and later they can incrementally learn about them.
This is the same way I learned programming, the people learn emacs and vim. You start with the MVP feature set that will make it functional and you learn about features from there.
Even AWS, Google Cloud, Azure, even though they're mega services of now insane size and feature breadth, they're creating single services that are easy to understand and compose together. You don't need to understand a lot to copy files in and out of AWS s3 for example, and you can compose many services together even after a basic understanding. You can slowly learn about more features as you work.
In the future, things like apache arrow will also help as you can retain services as modular but potentially share memory when composing services. This idea of service composition is also common in Haskell and other functional languages, where if it's possible to do lazy evaluation you can avoid redundant work. Now that the unix philosophy is being more widely adopted, although renamed to microservice fundamentals, these ideas of lazy evaluation and protected data sharing probably be the next level of services research.
I sure hope you are right.
Just look at kubernetes which wraps UNIX in a big complicated opaque 'opinionated' wet blanket.
It just puts an opaque YML fronted on a bunch of stuff I already know how to install configure and troubleshoot myself: firewalls proxies, web servers etc...
It dawned on me that people probably use it because they haven't done much sysadmin, but they want to deploy production systems.
As a sysadmin/programmer of 20+ years (what full stack used to mean I guess) it just mystifies me why someone would want to layer a barely documented front end on top of their infrastructure.
To me k8 feels kind of like a monster hybridization of cpanel & webshpere.
Containers are not programs. And even they were, they absolutely do not adhere to the Unix philosophy.
> Kubernetes is the operating system in this analogy
I've heard this enough times and seen enough K8s systems in production that I'm labeling this "fluff". K8s is an operating system in the same way that "everything is everything". K8s is software that manages containers and their resources. I think of it more like a virtual environment than an operating system.
From Wikipedia:
> An operating system (OS) is system software that manages computer hardware, software resources, and provides common services for computer programs.
You can generalize "software resources" as container resource limits. You can generalize "common services" as the network and filesystem layers. And you can generalize "computer hardware" as kernel configuration on the control plane.
All of these are purpose built abstractions built on top of the real operating systems. By that logic you could call any virtual environment or virtual machine an operating system. And if that's the true then the comparison isn't useful at all.
They are programs (barring some semantic nitpickery about "programs run in containers"), and like programs, we can't make blanket statements about whether or not they implement the Unix philosophy. Some do and others don't.
> I've heard this enough times and seen enough K8s systems in production that I'm labeling this "fluff". K8s is an operating system in the same way that "everything is everything". K8s is software that manages containers and their resources.
Yep, like I said, an analogy: "An operating system is software that manages processes and their resources". By definition, the operands of an analogy are different, so if your thing is taking any analogy and calling "gotcha!" because you found some differences, then I'm afraid you're missing the point.
> From Wikipedia: >> An operating system (OS) is system software that manages computer hardware, software resources, and provides common services for computer programs. > You can generalize "software resources" as container resource limits. You can generalize "common services" as the network and filesystem layers. And you can generalize "computer hardware" as kernel configuration on the control plane. > All of these are purpose built abstractions built on top of the real operating systems. By that logic you could call any virtual environment or virtual machine an operating system. And if that's the true then the comparison isn't useful at all.
That's quite the straw man.
Again, you can generalize and call them "programs" but the generalization renders the meaning of "program" in this context useless.
I don't see the strawman here. K8s and containerization is frequently mischaracterized as the operating system of the future. The technical details matter and speaking in generalities is misleading (IMO).
> There's a lot going on between a K8s pod, a container, and the actual program. It's just not Unix-y at all.
The Unix philosophy pertains to composition, not implementation.
> I don't see the strawman here.
You picked someone else's definition of "operating system", generalized it, and then attacked it for being too general.
> K8s and containerization is frequently mischaracterized as the operating system of the future. The technical details matter and speaking in generalities is misleading (IMO).
Yes, again, this is an analogy. If you interpret analogies as a guarantee that the operands are identical instead of simply "similar", you will be misled.
I'm still a VIM user to this day, but my learning experience was hardly a clean incremental process. VIM has no MVP if you're using it to program computers. Early on, I spent hours getting sucked into making the editor do what I wanted. I only stuck with it because I believed the investment of time was worth it (and it was).
> You don't need to understand a lot to copy files in and out of AWS s3 for example, and you can compose many services together even after a basic understanding. You can slowly learn about more features as you work.
This is dangerous though. You might unknowingly become a victim of vendor lock-in. Or you might have to throw out all of your work because you didn't understand the constraints of the system.
Subjectively, my evidence is how I (an experienced person) evaluates software and tools today. Step 1 is learning the architecture, Step 2 is finding the limits/constraints, etc (there's more but you get the idea). When I was evaluating Git I spent a couple hours researching the internals. This sold me on Git but it also made Git much easier to use down the road. It's not a very big time investment most of the time. This kind of evaluation has been so important when evaluating complex stuff like databases and SaaS products.
> Now that the unix philosophy is being more widely adopted, although renamed to microservice fundamentals, these ideas of lazy evaluation and protected data sharing probably be the next level of services research.
Microservices have been a disaster. The majority of implementations I've seen are brittle/buggy/convoluted messes. Complexity that was previously managed in code has been pushed to infrastructure -- and no one knows how to do infrastructure!
Most people that I talk to don't really understand what made the Unix philosophy work. Everyone cites "programs that do one thing and do it well". That's certainly the fun part, but it isn't what makes Unix programs so great. It's how programs do I/O (text and pipes).
In other words, it's not enough to build tiny modular single-use programs. You need an equally elegant system the combines the programs together. The programs need to strictly adhere to this system otherwise everything breaks down. Containers are not it. K8s is not it.
The simplicity of the individual services means it's easy to compose within a cloud, but it also means (particularly when the fundamental problem concept each is directed at is similar) that it is easy to abstract from a particular cloud by composing the solutions from separate clouds for the same problem behind a shared abstraction layer.
Example: How does AWS EKS provision control plane worker nodes? AWS CloudFormation. How do you manage security policies? AWS IAM. What kind of servers do you tune your software to run on? AWS EC2 of course.
Now let's say you wanna take advantage of this fancy "composing the solutions from separate clouds" thing. Let's say you like Azure's Blob Storage over AWS's S3. Guess what? You're now paying for cross-cloud latency! Both in time and in cost.
The devil is in the details.
Maybe the lesson is to figure out your minimum viable audience and build on the tools they already know?
Most other users need safety restraints such as a GUI. Doesn't stop you from using minimum viable product as a philosophy.
> A minimal viable program is the smallest program that solves a particular problem. It is small and beautiful. It has no additional features.
From the FreeBSD source tree (/usr/src/usr.bin/tr):
$ svn log tr.c | tail -6 | grep -v '^---'
r1590 | rgrimes | 1994-05-27 08:33:43 -0400 (Fri, 27 May 1994) | 2 lines
BSD 4.4 Lite Usr.bin Sources
Yet, when a node developer creates left-pad...
(1) Dependencies always add certain minimal level of complexity, so a too-trivial dependency can add more-than-warranted complexity as a dependency even when the implementation itself would not do so as an integrally-maintained component of the larger system.
(2) the nature of the npm distribution ecosystem (and this is by no means unique to npm) means that the complexity and risk added by dependencies managed through that ecosystem is more than it otherwise could be with immutable, append-only repositories of dependencies.
The include is immutable (unless there are macros).
Append only sounds like the included code should be evaluated after other code in a given file (so main defs would take precedent over included defs? Or main defs would be over-written by included defs... I guess it depends on compiler/evaluator implementation.
The more I type here, the more I realize how much I don't know about this stuff...
Legal and practical constraints would probably mean any real system would be at best an approximation of this ideal, but a big part of he leftpad problem is that NPM at the time didn't even try to approximate it (the response also underlined an NPM policy existing at the time that was useful in that context but dangerous in general, in that NPM generally allowed others to claim the name of abandoned projects.)
http://erlang.org/pipermail/erlang-questions/2011-May/058768...
> - contribution to open source can be as simple as contributing a single function > - there are no "open source projects" - only "the open source > Key-Value database of all functions"
Look at the examples he gives:
is_blank_line
is_prefix
remove_leading_whitespace
remove_trailing_whitespace
remove_leading_and_trailing_whitespace
skip_blanks
trim
https://news.ycombinator.com/item?id=8342755
https://joearms.github.io/published/2015-03-12-The_web_of_na...
There is absolutely no problem with the first one. Nobody is complaining about the author writing left-pad, only about it not being available in the standard library or some larger package. Changing the entire infrastructure so code becomes immutable and requires no trust would work too.
http://erlang.org/pipermail/erlang-questions/2011-May/058768...