In favor of small modules and plumbing
blog.webb.page
blog.webb.page
Now, I'm not inherently against having small modules for small tasks, but I'd much prefer if those are used like sprinkles on a more solid foundation. So a small Perl module on CPAN? Sure. Your whole web app consisting of small modules, turtles all the way down? No, sir, no.
It also means that you've got plenty of build systems, it's incredibly hard to change the output of all your dependencies to not-ES5, it's way more frequent that one developer introduces new dependencies, possibly duplicates etc.
Sadly there's no decent way out. JS is getting a new syntactic feature every second Monday, but no big std lib movement is out there that I'm aware of. Sure, you probably wouldn't want to fit the Go/Python/Java core lib into a browser, but for the backend it would be quite nice, and would make tooling for the frontend less fragile. (And to be honest, even frontends are approaching mid-tier desktop UI complexity; otherwise we could've and should've stayed with jquery)
I don't need to be able to reason about single modules independent from each other often. I need to trust the whole system enough to not lose sleep over it.
On any realistic sized project you already need tooling for handling dependencies in bulk. If you want to build everything for ES6, that should be a problem you solve once in your build/dependency manager, not per-depednency.
> it's way more frequent that one developer introduces new dependencies, possibly duplicates etc.
I actually disagree with this: if you have large frameworks then you're more likely to have duplication, because you'll have frameworks that overlap but don't completely cover each other. E.g. C++ projects tend to end up with 5 different string classes, because every framework you use will bring its own (because C++ relies on big heavy frameworks because it has a very high per-library overhead due to limited tooling). Small focused dependencies make that less likely.
And if it isn't included, why would 16 small subsystems have fewer duplicates than 2 large frameworks?
By the way, C++ is exactly the trajectory I see JS moving towards. A plethora of different sub-languages and library sets. Only with the difference that it's both easier from a tooling and culture perspective to bring in new tiny modules.
A language standard library is just another framework - Python's one is famously "where modules go to die", but we see much the same problems in e.g. Boost. A "good enough" solution in the standard library can be worse than nothing, as it means better alternatives struggle to gain enough momentum, and this gets worse as time goes by: functionality that's in the standard library has to evolve slowly.
> And if it isn't included, why would 16 small subsystems have fewer duplicates than 2 large frameworks?
Because the common things can be in their own small pieces. If a database-access library, a web templating library, and a config parsing library all need to do string manipulation, then if they all implement it themselves we'll see duplication, whereas if they all use an external library then we won't (this is after all the point of libraries). Maybe the ecosystem will have two different string manipulation libraries on the go with one gradually displacing the other, but that's likely to only happen when there's a genuine improvement being made. Whereas if everything's a big framework then eventually everything implements its own copy of the common utilities that just about everything ends up needing - collections, datastructure manipulation, all the really generic stuff.
When the left-pad guy had his snit, we didn't notice at all because we had a caching NPM mirror.
http://npm.anvaka.com/#/view/2d/merge-deep
All but one of those dependencies are from the same maintainer. Yet I see two major versions of kind-of, for-in, and lazy-cache. What's the point of creating reusable functions if you're not going to reuse them?
This is a huge waste. First on the filesystem, no wonder our node_modules folders are hundreds of megabytes big. Second it wastes bytes in memory and in CPU caches since they don't share byte code and each function has to be parsed, compiled, optimised separately.
I seem to be in the minority thinking this situation is insane…
Like we all know nodjs is slow(er than other things). The question is not how to make it faster by doing things like keeping dependencies up to date but how much there is to gain by doing so. I suspect very little.
The problem with lots of tiny modules is consistency and power.
Sure its "open source" but each one of these modules are in various states of decay. better to lump them together in sets of functions, so that more eyes can be concentrated in the same place.
The entirety of engineering is premised upon leveraging powerful abstractions; cf. the reason we're not still coding via electric charge.
Edit: To expand on the analogy: "A two-person startup already uses twenty-eight other tools" [1].
I assume you're talking about userland programs such as "ls", "cut", "grep", etc. ?[1]
But C Language is also part of UNIX and programmers typically link against a large more comprehensive "standard library" such as glibc:
libc.a, libc.so <-- includes printf
printf.o
strcat.o
strncat.o
malloc.o
leftpad.o
... <hundred or thousand more .o files>
[1] https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_a...
I disagree. I personally find that with large projects, small dependencies actually become much harder to reason about - they sort of get lost in the codebase. Not to mention the duplication of small packages on npm doing similar things - with a large team, it's easy to add both to your codebase and then we're in a new sort of duplicative mess.
But this would be a functional problem, irritating at best and a potential health hazard at worst. Using plywood for the back of a chest of drawers is no such thing.