Pika/web: Web Apps Without the Bundler
pikapkg.com
pikapkg.com
The reason that JavaScript build tooling is so complex and introduces so much overhead is that the ecosystem's culture is fundamentally broken. Every single one of these tools is a raging dumpster fire because it's built on top of all of these layers of low-quality interdependent crap.
I'm increasingly moving towards module-type script tags and standard ES modules everywhere, with no build step during development. It's still challenging to use node-targeted modules in this fashion, so I really do appreciate that people are working on finding ways to make it work. I just wish that we could do it without pulling in so much third-party code and the large surface area for failure and security problems that go with it.
Just trying to understand, is this a bad thing?
Someone else made an open source CLI spinner library which also uses other people's existing open source libraries. This saves a lot of time and gives developers many good options.
Should Pika write and maintain its own custom CLI spinner animations? Are you saying the CLI spinners should be standardized in the next version of ECMAScript itself?
How is this worse than the same thing written in Python, for example? (I mainly use javascript, so maybe I haven't been exposed to the kinds of alternatives you're thinking about.)
The real question is: do you -really- need an external lib with 20 dependencies just to show a freakin’ loading spinner? Remaking the wheel is bad but so is never making truly simple things yourself, or just not using them.
What happens when a common package breaks? What happens if it gets hijacked and becomes a security vector that’s impossible to spot because it’s loaded as the 567th package in a dependency tree?
The answer here is to have a strong stdlib where do you don’t need to pull in 3rd party packages all the time for trivial things, and not including a million small packages in every single project.
So the problem is the sheer number of dependencies? What is a reasonable upper limit?
Yes, javascript should continue to standardize commonly used features, but avoiding dependencies doesn't seem to be a solution.
If anything, more dependencies are a good sign because they imply that other people have spent more time and effort on a solution than anything you'll be able to hand-roll for single-use.
It sounds like the root issue here is just dependency management. If our package managers were solving this issue well enough, there should be no practical difference between 2 big dependencies with significant functionality (and more code to review) or 20 tiny, easy-to-review dependencies.
We don’t allow automatic upgrading of packages/dependencies due to the risk of malicious code making it in (see https://www.npmjs.com/advisories for examples). Yeah there are companies that will help manage your vulnerability process but it’s still a lot of overhead and only grows as the number of dependencies grows.
There’s also the whole left-pad mess from a few years back which shows you always need local archived copies of any dependencies you use.
That's a good idea, how do you do that?
> you always need local archived copies of any dependencies you use.
Are you committing your dependencies? Or using a package manager with caching?
Off-topic, but can you write about how you manage this without tons of manual work?
This preferably means reading the code you're pulling in but it's unrealistic that we're going to read 2000 constantly changing dependencies for every deploy, so you need to establish trust some how.
Reputation of maintainers, dep CVE scanning, SAST and protective monitoring can all add additional assurance, but they won't protect you from a random hijacked npm module, and the more you include and from more people, the more likely you'll be affected by a zero day.
Having an entire community depend on tiny libraries that do nearly nothing exaperates this problem, and if you use something that almost nobody else does, you're unlikely to be saved by npm audit.
I don't use node daily driver, but I assume npm audit growing to include reputations and frameworks owning more of the dependency tree will help, but the users of the system also need to be considerate of the risks they take and the trust they place.
Another reason why I stopped hosting dependencies in SCM is native modules. I wish Node.JS itself could become a platform layer so that I wouln't have to use these native modules.
Another thing is compile to JS, where a tiny change in the source might cause a huge diff in the generated JS.
It’s horrifying from an operations perspective where the job is to make sure everything works.
Developers can afford to ignore looking into dependencies, operations need to make sure every dependency is functional and safe.
If you write a piece of C# using the standard .Net library you can be fairly sure it’s safe and sound. If you write something using 2000 JS packages, you have to read through every one of them to be sure.
Putting the onus on the developer to do a good job with regards to secure development practices is an essential part of a wider system.
This is so incredibly important because the main development of the language is so poorly handled.
I think that modern javascript is pretty good and I'm grateful for the syntax extensions that have effected that, but the standard library is crying out for some love.
This is what a lot of people love about Elm: it's a lovely language with a first-rate standard library.
The Elm ecosystem is smaller than the JS ecosystem, but of course that's a hard requirement of a nicer foundation.
More info: https://elm-lang.org
I still have some array helper methods just to shuffle array or randomly pick one.
I think it's time to put everything in future JS.
I don't disagree about the security issues, mobile performance, and energy consumption though, but as a dev I preferred the experience of writing AS3 apps by a long shot.
I use compile to JS languages when I can, but not sure what now you'd want for a frontend JavaScript standard library beyond what's available in ES6.
Yeah TS has many of those, but TS is not JS and it introduces new problems.
Not every god damn webpage has to be as complicated as Gmail.
Do you know of any listings of third-party ES Modules that are designed to be used directly? i.e. their full URLs being present in import statements that the browser sees at runtime. Even toy stuff would be interesting to look at.
and then what do you do for production, webpack?
I can get away with not supporting Internet Explorer, though I understand that it's not a trade-off that everybody is comfortable making.
Then we test the website on different browsers (we all work with OSX), FireFox works (of course), Opera works (is Chromium), Safari, naah, it's the new IE. We do some minor fixes for Safari.
When we send it to the client, nothing works, of course the client uses IE so we need to rewrite a lot of stuff to make it work for "most" people. (I live in Belgium, there are "a lot of" IE users still. And even if IE is not widely used, our clients always use IE :( )
But wait, why rewrite our application, this can be fully automated! We run our code through babel/webpack. It automagically works everywhere!
Now I understand there is some performance penalty by using transpilers, but using them leaves our code mostly bugfree on all browsers and the client is happy.
The big problem with those fancy new features is browsersupport. If every browser followed specs correctly, we wouldn't need these tools. They make the job of the developer easy, the bosses wallet full and the client happy.
There are really two solutions to this: either avoid using js dependencies, at which point you won't need npm or pika or whatever, or use bundlers.
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
It's non-negligible for "thousands of dependencies", of course, but nobody would be doing that. "Thousands of dependencies" is for library development, not for library release packaging.
NPM (or Pika) is still useful under such a paradigm, for the same reason Cargo or Bundler or Pip or Mix or whatever other tools are useful: these tools let you specify your direct dependencies using symbolic version constraints, and then retrieve+update+lock those dependencies. This is helpful whether your direct dependencies are "baked-down" release packages or raw deep-dep-tree source.
Sure you can fallback to local copies, but that just means your app hung there for X amount of time, and the added complexity of adding fallback logic.
Plus if you look at the size of these libraries, especially utility libraries (date-fns, underscore, etc) they are huge, and you usually only need a few functions. Relying on cache will never be a bigger benefit than tree-shaking.
Sure you could manually try to extract those functions, but that's a much much worse developer experience.
So glad this was such dominant practice and advice for the last 10 years. I wonder for how many of those influential practictioners have known it's more of a lazy include than clever cache reuse. Cargo culting writ larger than most.
If you are talking about products with lot of dependencies then of course but don't forget that bundler situation used to be completely different. There is not that much of difference (besides number of requests) between concating libs in sequence and loading libs in sequence.
I just checked, and googling "jquery cdn faster" brings up plenty of examples.
This is not really true for SPA's, but I hope that's a trend that will eventually sizzle out again except for specialized cases.
Just grabbing what you need can have significant benefits. I agree with the latency problem though, it would be nice to have a HTTP/2 or HTTP/3 server that has a basic understanding of the JS dependency tree and preemptively push the entire tree.
1. Typically, you only ever care about minimizing bundle size in cases where latency is consistently questionable
2. In those cases, whatever UI you're supporting is likely then not meant for latency-sensitive operations, and thus the overall workflow tends to have low state complexity
3. Just how much actual value is added in those scenarios by having the ESM workflow specifically in conjunction with importable NPM modules?
I'm sure you can think up some contrived use cases but I'm just not seeing any particularly compelling ones.
I suspect we'll see the opposite, with people building desktop type applications and deploying through the browser (especially with the growth of wasm)
Sure, if you only use ES5 then you don't need Babel or Webpack - but devs are tired of this shit and really want to use ES6 or Typescript.
So, "nearly impossible" is an exaggeration, but "would rather gouge my own eyes out" is perhaps not.
Everything else is programmers coming from other languages and believing that actually learning javascript is beneath their dignity, and persistently insisting javascript be more like whatever their fave is.
I will say this though about build systems/bundling systems in general: Combined with linting and type checking, it can stop you from accidntally shipping a large class of bugs/mistakes. An out of place semicolon breaking a build is a better outcome than breaking a production website.
We switched our whole codebase to <script type=module> and it's such a joy now.
Never going back to bundlers. Never going back to having a compilation step on the server.
* You are hosting an internal enterprise application and you are forcing all of your employee desktops to use Chrome any case.
* You are not serving a web site, but a business web app, which only works on a modern browser - because it is a business oriented app people are willing to switch the browser to use it. E.g. 3d modeler.
But for srs, couldn't I achieve more or less the same effect by not putting node_modules in my .gitignore and then making aforementioned modules accessible via static content dir pathing?
Sure you can fallback to local copies, but that just means your app hung there for X amount of time, and the added complexity of adding fallback logic.
Anyway, committing -- and directly serving -- all of a given project's transitive dependencies sounds like a tough sell; without decent tooling to dedup and tree-shake, it might go from inefficient to implausible.
my bad
The "simple" project is a bunch of files ... with no instructions on how to use it.
Although at that point, where your app needs to reliably run in both Node & the browser, there’s a good case to be made for bundling. All that complexity is giving you something valuable, which is the point that the article tries to make:
in 2019, use bundling because you want to and not because you have to.
It wasn't that no one had thought of showing the dumpster fire (to use a quote from a earlier comment!) but simply that the tech wasn't there. A lot of the problems here are of the same nature. The right level of tech is yet to built and hopefully when it is it may also improve the nature of the media that gets bundled (much as telecine was pretty poor looking when compared to the video tape recording that supplanted it later). So trying new tools is good. But I haven't seen that paradigm altering shift yet.
Browserify does one thing and does it very well.
I still would like to not have to use any of these, however.
I read through the linked page and it's not totally clear to me why you can't just import directly from node_modules. What Pika/web appears to do is find node_modules that use valid export syntax and copy them to a different dir. Is that correct? Am I missing something?
Bingo
This is why it doesn't fully pass the smell test for me
So I basically don't exclude node_modules, and set up a simple webpack config to suck in my deps and output them as ESM modules to a given static dir of my choice
A neologism that I can get behind.