Some notes on using esbuild
jvns.ca
jvns.ca
Being able to say "I don't know how this works" requires a lot of experience too, not just confidence.
Especially in an interview, people saying "I don't know anything about what you asked", depending on the requirements, can end the interview but could also make the candidate gain a lot of points.
Thanks for the write-up!
There are new esm based cdns (similar things have been around since the browserify days), awesome new build tools like esbuild, skypack, and swc that are faster and/or simpler than webpack. new language features have been standardized that reduce the need for various types of libraries, etc.
For anyone who doesn't spend an unreasonable time following these developments discovery of all this cool new things seems very tough.
On the opposite side of things, as a developer of a library I think is really useful, figuring out how to help people find it, I have no idea where to start.
So I end up doing the same thing the author was doing: just use <script> tags. If I need a library I try to download a minified version of it and put it in a script tag. This is usually good enough for me as I typically use JS for very small things. “pretend it’s 2005”. It works and if something breaks I know how to solve it.
For the kind of JS projects that I work on, learning the tooling can easily take 3X the time I need to write the project itself. I know that there are several meta-tools that will automatically configure all other tools for you, but I really dislike them. They usually end up downloading half of the entire internet, and when you move an inch from their preferred defaults everything becomes a nightmare.
Esbuild seems like the perfect tool for people like me. It's simple, fast and it allows me to import things in JS! I'm definitely going to look at it next time I'm writing some frontend code.
Agreed. I wish `npm install` was just some option for those who wanted it. But no! Somehow it became the defacto standard and everyone just expects everyone else to already be onboard.
I got onboard reluctently because that's the only way to use the libraries I cared about.
> problem 2: I don’t understand frontend build tools
Absolute and total agreement.
I can't agree enough.
I think no one really understands them.
> “webpack” which I will not try to explain much because I don’t understand it, I think it’s a system with a million plugins that does a million things
I haven't worked with anyone who understood webpack. They just use a scaffold-builder to set it up in a pre-configured way and then you just never touch it.
> Recently I learned about esbuild which is a more Unix-style tool.
esbuild is the best thing that happened to frontend development. It finally freed us (well, me anyway) from the messy js build systems.
But I'd like to strongly object to describing it as a "unix-style" tool. Unix style tools to me are tools that deal with plain text and expect you to pipe them with other tools to get the job done. esbuild is nothing like that. It's the entire build pipeline in one tool. You don't work with it by piping its inputs and outputs to other tools. It provides you a programming interface and you write code to make it do what you want.
EDIT:
To clarify, I think it's a very good thing that esbuild is not a unix-style tool. I know the common wisdom on HN is in favor of the unix philosophy, but I don't buy into that.
Your definition of a unix tool is a strange one.
If a tool does everything for you from start to finish, it's not following the unix philosophy.
Unix command line tools are basically meant to be like small library functions in an ad-hoc text-based interpreted language.
Commands are supposed to be focused on one specific task.
More complex tasks are supposed to be achieved by orchestrating several commands in the right order.
The interface between commands is a stream of text.
https://en.wikipedia.org/wiki/Unix_philosophy
For example, one of the core items in the list:
> Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
But as you can see, esbuild is nothing like that. It doesn't expect itself to take input from other programs, nor does it expect its output to be piped to other programs.
Another formulation:
- Write programs that do one thing and do it well.
- Write programs to work together.
- Write programs to handle text streams, because that is a universal interface.
Again, esbuild conforms to none of these points.
I actually don't. esbuild not being unix-style is a good thing in my opinion.
Is webpack really considered such a difficult beast? I was an early adopter of TypeScript so I think I started with grunt, then gulp, then finally webpack 3, 4 and now 5.
I mean it's not my main job or anything, and I don't really enjoy configuring things but spending a handful of days reading the docs once a year or whatever is not really a problem. I have a couple of files, one for local one for production. About 50-70 lines each and considering it is compiling typescript, linting, minifying, scss, testing and so on... I find I get a lot of bang for my buck. It's not like the previous tooling was really any easier to deal with.
The production build takes about 50 seconds, and the local build takes about 3 seconds. It's honestly the least of my problems.
For whatever it's worth, I'm the sort of person who writes makefiles for fun, and I regularly work with the build systems of go, C, rust, java, ruby, python, and javascript.
webpack is easily the most complicated, slow, frightening build system I have ever encountered and I will go far out of my way to avoid it.
If you really understood the previous tooling, that probably explains why you have the mental models in place to understand what webpacker is doing with increasing automation. Say, if you were using babel without webpacker already.
For people who skipped that or came to JS in webpacker era (myself included), it's hard to understand what is going on. the universe of what webpacker does seems inconsistent and random and involves concepts underlying concepts underlying concepts where it seems like you need to understand the mysterious base to understand what the top-level tool is doing before forgetting it and letting the tool do it for you.
But webpacker isn't just insane, it is what it is for a reason, involving solving specific problems that were in existence in previous ways of doing things. If you had those problems personally... it's a lot easier to make sense of it.
I assume/sense. I don't know this from personal experience with webpacker... but it's a familiar sort of situation with software development technology.
I think this a big issue for non-js / frontend people, particularly in the python webdev community.
People don't know the story of front end and the user demand for refined interfaces. The demand has been high even though browsers just recently offered consistent support for modern js / css features.
I suspect this will come back to bite backend people, to some extent because the dust is settling on frontend. So the anti-everything on frontend attitude might blind people to the rapid advance of JS and node.
I guess it's better since version 4, but when you need to change/update your build script, you brace yourself.
To be fair I use esbuild on a Phoenix project right now and some things are not really that much clear on how to do them.
Like yesterday I wanted to put eslint in the "watched" build process, and I'm still not sure how I would do that.
At least it's fast...
Sorry for this, but I find the parallels a bit funny.
Moreover, webpack does all sorts of things I don’t care about: support for a dozen different module systems, javascript versions, polyfills, scss, JSX, etc etc. I just want to issue a few requests and update the DOM.
I guess I’m spoiled by Go and Rust where I can basically just throw a list of dependencies at the build tool and it spits out an executable (almost immediately in the case of go).
Config files are brittle and very hard to understand.
Spending several days sifting through the webpack docs and maintaining 70 lines of config files sounds like a nightmare.
Now, for esbuild, I have more lines than that for the build script, but the thing is, it's just code. The configuration part is about 10 lines. The other stuff is just programming logic.
Navigating documentation to understand what parameters something accepts is ok.
Spending time editing code is ok. It's what I do most of the time anyway.
It's tractable.
A web of configuration files is not tractable.
Not sure how much that plays to my strengths though because everything I have ever used has json file configurations or object file configurations.
Anyway I am definitely going to kick some tires on ESBuild, but I will just wait a while so thanks for some additional insight.
Back in the day I was using amd.js and they had a scriptable command line utility so I just wrote a javascript file that collected all the required inputs for it and then built up the build command. (or something like that .. I don't remember the details).
I don't think you realize how hellish this sounds. I haven't had >100 lines of build configuration in any other environment since my configure.in/Makefile.am days, and even that was more transferable to other tools than Webpack, which is only good for Webpacking.
And there are plenty of higher-level tools than webpack that are pre-configured to handle most of your variations anyway (eg CRA) and are a much better starting point if you don’t need to fine tune your build or do complicated/exotic things (which isn’t a breeze in any language I’ve experienced).
This is another reason to favor small and simple tools that do one thing: you can “take it apart” and understand it by using another simple tool. (Simple in a relative sense, e.g. compared to Webpack).
By the way for those who don’t know what strace is, it’s a command line utility that prints out system calls of another process, like opening files or network connections.
I did try various setups but none worked. With Vite at least I was able to alias libraries to same import thus avoiding importing them twice/x times.
But I'd kinda prefer using esbuild as Vite can get funky too, especially I've noticed if you build a local library with multiple chunks (eg just using tsc) the HMR will get out of hand. As I just tried, it basically seemed to go on indefinitely. Maybe there's a config for that but it's not nice, I can tell you that.
I would say it takes a bit more of work because the tutorials are normally written on commonJS (require), but otherwise it's just a different kind of work. Both of them have their peculiarities and issues, and CommonJS might also bite you big time. So if you have to choose, and both give you a bit of pain (for now), I'd suggest to choose the one that is more future-proof.
About the article on Esbuild and ESM vs CommonJS, some other personal notes:
- Node.js' `import` is more strict than frontend tools' `import`. You can do `import myfile from './myfile'` on the front-end but you need to specify the ".js" on the backend.
- You can no longer use __dirname with ES6 modules. That's a feature, and I found that I normally have enough control over where the script is run from that I just prefer write all relative paths instead of bothering shiming it.
- The Esbuild from the author is great and I use something similar for simpler projects and libraries I develop. However for making WebApps, I prefer to also have Hot Module Reload. It's like Prettier, at the beginning it feels unnatural and overbearing, but after you get used to it you realize what a huge order of improvement it is vs the way you were used to.
Drove me crazy until I realized in the full path there was, early in the path, a single "%20" and my JS senses told me it doesn't make sense for spaces to be url-encoded in paths.
In which ways? I’ve always found CommonJS to be extremely unsurprising. What will bite you is the tooling built to interoperate with ESM.
- http/2 multiplexing has made it unnecessary to limit the number of script or css files loaded at once. Faster connections together with http/2 mean payload splitting is no longer necessary for js and css (which means everything can load up front from index.html). Gzipping removes the need for minification.
- Any npm package can be loaded as script tags or through direct es6 module import from unpkg.com. I use a single ‘externals.js’ which exports those dependencies so I can just do “import { foo } from ‘externals.js’”
- sass is mostly unnecessary thanks to widespread css3 support, @import for modularity (see also http/2), bem notation for namespacing, and css variables for reuse
- es6 is supported in all browsers, so you can write modern code. ES6 modules “just work” if you import from a file path.
- vue and preact are designed to be used without build tools. An app deployed as a static site can use hash routing to need no builds and no server-side router.
Experimenting with this stuff has really made me wonder why we use all of these build tools.
How can you resolve a module name to its path without external dependencies?
_need_ is a strong word. You don't _need_ gripping either. Both minifying and gzipping still provide substantial benefits.
> Any npm package can be loaded as script tags or through direct es6 module import from unpkg.com
This prevents tree-shaking and dead code elimination.
As for tree shaking. When using lightweight libraries meant for the no build path this isn’t necessary because these aren’t pulling in lots of dependencies. Preact is tiny and is a good enough react for my needs.
I wouldn’t recommend this approach for large web apps, but most web apps are small enough that a no build tools approach seems viable to me.
So is React. JSX and React are 2 different things. But honestly, React without JSX is a pain to write.
Minification and hashing for secure script embedding are still needed, but it could provided directly by a server middleware, depending on the language one uses.
IMHO Typescript is still something useful.
I find typescript useful for libraries, less so for application code.
I love this and maybe it would be great for someone who works on the spec/bundlers to write a "Dear Julia" letter explaining things and giving context to specific parts on why things are the way they are, and pointing out clear ways that we as a community should try to improve.
Knowing nothing about Java, I wouldn't expect to open a Java file and know what every line means. There's a million reasons to find modern web dev complicated but not taking the time to google "js import" is not one of them. Also I wouldn't expect to "just know" how to take a Java program and build it in a way that I could host it somewhere, I'd expect to have to work at it.
I consider myself having deeper knowledge about JS and node than any other language, having spent more total time in it than anything else (except maybe bash).
I have a clearer understanding of what import/require really does in most other languages.
The flow-chart complexity and possible outcomes of what `import` really does in JS/TS are certainly greater than anything else I've encountered.
(I do find go's repository-based approach quite frustrating to work with when forking dependencies but at least it's straight-forward)
If I understand your point correctly, then that's a non-issue since modules are here.
In the go.mod file you have the current module name, and all imports of yourself will use the current repository, instead of calling the original.
So you fork the repository, but keep the original name in the go.mod file, then you don't have to modify any imports.
https://golang.org/doc/modules/managing-dependencies#unpubli...
I still find the whole gopath/gomodules thing unfortunate but it is what it is at this point (and frankly that’s a bit out scope since we’re talking imports here.. if we’re talking about the state of packaging then JS absolutely has competition; e.g. https://xkcd.com/1987/)
There are multiple historical module systems, various official and unofficial syntaxes, and also the require keyword.
It is an absolute mess. I don't know anyone who understands all of the various flavors.
Yarn blew NPM away on speed with more features and nicer developer workflow. Webpack is on major version 5 and still everyone just uses CRA rather than try to configure it by hand, but then nobody really knows how to debug CRA if something breaks. The entire babel stack is ridiculous. The dominant tools have just been so awful, for so long.
I dunno if it's still true, but for years into people saying "LOL use yarn, it's a better replacement for NPM" it was still really easy to find packages that broke under Yarn because it lacked some feature or other, or was skipping some obviously-a-good-idea sanity check that NPM did and so crashed rather than proceeding after adjusting its approach, or to venture slightly off the happy path of doing "yarn install" and running into features that yarn didn't support, but NPM did. A venture into the issue tracker in that time period was enlightening, and I don't just mean the sheer count, but digging into some of the issues and why they were happening.
[EDIT] in fact, in the agency I was at at the time, a kind of joke developed that a project wasn't fully underway until you'd been forced to replace Yarn with NPM.
I was talking about import in JavaScript itself, not build tools or package managers.
Within the actual language, there isn't a universal syntax just to use a package. In Java, there is.
There is. The issue is one of mindset. You recognize Maven and Gradle as belonging to a sort of "parastandard" set of technology, offering proprietary (albeit free/open source) glimpses of how one could conceivably solve the set of problems that they're meant to be used for, but when it comes to JS, you're elevating the parastandard stuff to the level of being part of "JS".
Neither CommonJS nor NodeJS's `require` nor package.json nor TypeScript nor esbuild are part of JS. You shouldn't give them any privileged status that you aren't willing to give to Maven or Gradle when you think of Java.
It's entirely possible that the projects you're most familiar with aren't using them, but the JavaScript language has native modules, including standardized syntax and semantics for import statements.
Most importantly, there can't/won't be a breaking change for the previous iterations of module import, so we will continue to have many different ways to import.
It's not an issue of whether JS has a "blessed" way of doing it (and it didn't even have that for a long time). It's that there's more than 3 ways to do it, and many of them look similar and have confusing conflicts with compilers/packagers.
The four environments I often end up with are: Browser, Node.js, Rollup.js, and TypeScript.
For example, for a long time now, it has been sensible to just use "import" statements in code you intend to run in the browser. For a debug build, you don’t have to bundle, since browsers support import, but the imports have to be relative and have the full pathname. Node.js has a configuration option that allows you to use "import", but it is finicky and it can be surprising how many things will break when you turn it on—the alternative is to use ".mjs" as an extension. TypeScript has specific requirements about file extensions as well, and does not modify them. Finally, Rollup.js is hopelessly configurable.
Consider a simple requirement, “This code runs in the browser, I want to write a test using Jest.” I know it can be done, but I’m spending a bunch of time digging through forum posts to make it work, and understand how to make Jest load the code that already runs in the Browser, because the Node.js environment is so different… and most of the time, I end up writing code just to get something to build.
So the more than 3 ways, off the top of my head:
- Import paths: non-relative? relative? relative with leading slash?
- Import paths: .js suffix? no .js suffix?
- Files: use .mjs / .js suffix? use .js / .cjs suffix?
The confounding details you bring up are entirely the result of that idiosyncratic tooling. JS itself has `import`, thus there is no problem of the sort that you describe—at least not as you've described it. (I.e., this isn't to say that there are no problems, only that the way that your characterization of the problem as being one with "JS itself" doesn't match the problem that actually exists.)
To say it again, if you see a problem with JS, that's because you're elevating the parastandard stuff to a higher level than it deserves. If you're able to meaningfully separate "Java" from "Maven and Gradle" in your mind, then you should be able to do the same with "JS" and the NodeJS derpitude.
Moreover, you can skip them entirely and roll your own in a couple of hours [1]
Meanwhile in JS world? Build tools now change faster than frameworks, and not a single person is concerned with upgrade paths, or working in tandem with other tools.
And yes, import story is a mess.
I had to read the MDN guides multiple times and still need to refer to them.
Setting the build system part for my new projects is still the most challenging task.
Why do I have to learn how to build a grand piano every time I want to play a simple melody?
require ‘thing’ in Ruby requires the thing and that’s it.
require in JS should be use i. Node but not in the browser if I remember? Or maybe not? I don’t know, I guess I’ll just try things y til it works.
For all the other failings of the Python 3 migration, the Python community strongly and rightly rejected 2to3 which would've eventually produced a babelesque mess if continued.
NPM, import, etc. seem mindbogglingly complex in comparison, with an enormous number of ways to do each thing, most of which require at least one build step <script> just doesn't, and enormous numbers of third party dependencies, and if you go away for a few months and come back, there are good odds things will be different and broken.
Like Julia, I don't understand JS build systems and packaging etc., which no doubt contributes to the above experience, but I'm comparing to <script>, whose simplicity is almost unbeatable.
The most simple example is almost never a representative example that can be used as a standard to hold the whole language to.
In the same vein, one could say: "See how easy it is to build a C program with `gcc main.c`? Why should I have to figure out how to use make?". And the answer is simple: Most projects that go beyond a toy example have much more elaborate requirements(/whishes), like "I'd like to reuse code that another person has written in the past so I don't have to do it".
> shipping dozens of plaintext source code files over the wire that then might get executed in an environment you have no control over that doesn't actually support the code you wrote
What does "plaintext source code" or "over the wire" do to distinguish this from "compiling a binary targeting a minimum supported ABI"? Bundled JavaScript doesn't even have to deal with dynamic linking!
I'm not sure what I am supposed to do with that statement/answer? It's obviously very easy to just assume that every js developer must be an idiot but that is hardly a fruitful discussion to have.
> What does "plaintext source code" or "over the wire" do to distinguish this from "compiling a binary targeting a minimum supported ABI"? Bundled JavaScript doesn't even have to deal with dynamic linking!
Correct me if I'm wrong on that but it doesn't really matter if something is written in Go, Rust 2015, Rust 2018, Rust 2021, Zig, D or whatever else comes to mind, assuming static linking of course. I can compile it, I can ship it and the binary will work. I can't just ship typescript out, browsers don't understand it. I can't just ship modern js out as I have no idea if the users browser understands the code. Bundled javascript doesn't have to deal with dynamic linking because it is, in essence, static linking. The whole dynamic linking thing was sort of tried with CDNs shipping js libraries, didn't really work out all that well in practice, relying on some different service to be available for your dependencies is only a good idea until that service has downtime.
In this scenario, the binary is your ES3 or ES5 or what have you.
Compiling Go to an x86 ABI, Rust to the same x86 ABI, C to the same x86 ABI, etc. is an analogous problem to "shipping dozens of plaintext source code files over the wire that then might get executed in an environment you have no control over". It's harder, by order of magnitude and on every axis, than compiling TS, ES2015, whatever, all down to some lesser ES version. And yet these compilers are faster, more effective, and easier to use.
You've missed my point about static vs. dynamic linking entirely. The point is that C compilers did "bundling" for years while supporting dynamic linking, and now you're trying to say JavaScript is tackling some way more difficult problem when it has a much simpler language-to-"ABI" translation process and doesn't even need to support that?
As you say, this maybe isn't a fruitful discussion - but it's a true one. JS tooling developers got seriously deluded about the novelty of their problems and quality of solution at some point ca. 2014, and we will be dealing with the results for another decade or more. At the very least let's start being honest about it.
I also think that invoking "C compilers did X" weakens your argument because the C build ecosystem is quite a lot worse than the JS ecosystem and without any good excuses (its ecosystem is not rapidly evolving, there is no distribution model putting comparable pressure on latency, etc).
But in general, I agree that there is a lot the JS world could learn from the native world.
Other than module packaging (and that's admittedly a big "other than"), I don't agree. Esbuild is the first tool I used that's even close to e.g. Make in comprehensibility (~ ease of use x flexibility x ability to debug when something goes wrong). That's a pretty low bar, and I don't know how long esbuild will stay that way.
"Stockholm Syndrome" is the only way I can rationalize people vouching for C or C++ (or JS, to a significantly-lesser-but-still-not-negligible extent) build tooling.
Ignoring the nullified value proposition, there are other reasons besides (like security) to reject what the NodeJS/NPM world considers standard practice.
I may simply have got lucky so far with that but it's worked really nicely for me for quick things where I don't care about bundling.
I feel like JS gets thrown under the bus in discussions, but no body talks about arcane file structures in Java or implicit pathing lookups which have the same kind of "historical path dependence" to why they exist as JS has for require/import.
All languages have their warts and 99% of the time they're all doing the same thing just in slightly different ways.
Just be glad Gradle hasn't entered the JS scene yet:
With JS there is at least some standards now: both Angular and React at least has some systems for getting things running in a standard way and keeping it somewhat aligned.
With Gradle however I haven't seen two similar setups so far. Edit: I might have seen two similar setups.
I would say most small projects (especially OSS) tend to stick to a simple subset of Gradle even if it's multiproject build etc.
The amount of churn in the JS build tool space easily outstrips any one time learning you need to do to surpass the Gradle knowledge cliff where all this complex stuff just makes sense.
Android Gradle Plugin: here, hold my beer.
https://developer.android.com/studio/releases/gradle-plugin?...
I find working with Gradle infuriating. It's layers upon layers of magic. When something breaks, you get a baroque error message, a usually useless stack trace, and a suggestion to re-run the possibly very long build with -info or -debug. At that point, instead of not having enough output, you're usually drowned in megabytes of irrelevant messages.
Not a fan of all the magic behavior at all.
Gradle is incomprehensible until it isn't. It has a very steep but short learning curve and understanding is completely binary. It's very unfortunate but it is how it is.
I still take it over the JS stuff but I do acknowledge it's shortcomings. I also don't do Android development so I don't know if it's particularly bad there.
In general, I find DSLs make my life harder, not easier.
Gradle is my least favorite build tool ever. It's just too much magic in too many places.
This is not an endorsement of the JS ecosystem either. It also sucks, but it has wasted less of my time than Gradle (again, in the context of Android).
Useful and clear. Love it!
EDIT: think I found it https://github.com/webpack-contrib/terser-webpack-plugin
Unfortunately, my project is still on webpack v4 so I cannot use this until I update.
Kind of, but not really. There are JavaScript modules as defined by the ECMAScript standard and implemented in Web browsers and anyone else who cares about implementing the spec, and then there is NodeJS, which has its own way of doing things, along with people who refuse to give it up and insist on dragging those headaches into places where you should not otherwise expect to see them. People call the latter "CommonJS modules" (including e.g. the TypeScript team), but in fact CommonJS is a whole thing on its own that NodeJS doesn't actually follow. The thing that CommonJS and NodeJS-style imports have in common is that both used the `foo = require("bar")` convention.
To understand the mess that is the current JS ecosystem (and hopefully how to get out of it), people really need to come to grips with recognizing NodeJS's role as an IE-like entity:
- 800-pound gorilla
- not under the control of the best and the brightest
- implements its own share of proprietary (vendor-specific) shenanigans
- the people who shamelessly target it and see nothing wrong with doing things this way are the source of the the worst problems within the ecosystem or on a given project
- responsible for a massive corpus of legacy junk that you are expected to be familiar with and that it will take years to clean up
I think in about 6 months not touching common js modules from front-to-back for a whole project will be viable without major quirks.
But I was under the impression that node could generally import and handle strictly conformant CommonJS packages just fine in practice.
Of course, I won't argue that many npm packages not being conformant CommonJS packages makes things difficult for non-node server-side javascript environments, since they either need to add additional node compatibility, or might need their own package repository.
FWIW, "import react from 'React';" works fine with esbuild with no config, so I'm inclined to think this is a problem in Vue.
You misunderstood, they say this is exactly how it works. But they wanted a specific build instead of the default one of vue (the one with the runtime compiler, not default because it has performance penalties).
If you don't have the runtime compiler, then that means either you: 1. handwrote the the compiled templates, which is rather unlikely, or 2. you are using additional tooling to precompile.
If you are using additional tooling to precompile then that tooling ought to handle aliasing to the runtime only version, (or if not using a cli style tool, the instructions for setting up the bundler to use a loader should include setting up the alias).
However it is probably too late to fix things, since existing projects will assume the default is the compiler-less runtime, and would perform worse if things were changed, since existing projects would not have the new alias to run-time only version in place.
[2] https://svelte.dev/blog/sveltekit-beta#From_Snowpack_to_Vite
> But I stopped using those tools and went back to my old <script src="script.js"> system because I don’t understand what those vue-cli-service and vite are doing and I didn’t feel confident that I could fix them when they break. So I’d rather stick to a setup that I actually understand.
It's actually the experienced mindset.
With experience you learn to distrust complicated setups.
A beginner would just trust the tutorials and follow them to the letter without bothering to try to understand everything.
With JS build tools, I eventually gave up and accepted being uncomfortable.
(And to be clear, we're talking "levels of abstraction" here. Or interface/implementation. I don't need/want to understand the implementation of the tools/libraries I use. I need/want to understand how they work at the "interface" level, the mental model for what they offer and how to make them do what I want, and what choices are available on what axes. I normally achieve this. With webpacker, I give up and just copy and paste magic phrases).
Esbuild is a fantastic tool, but the nature of the problem domain is indeed inherently complex. "I don't know what import does" may sound like admitting ignorance, but it is also borderline clairvoyant. Resolving a module involves the incredibly convoluted node resolution algorithm, semantics of doppelganger packages, semantics of peer dependencies, architecture-specific optional dependencies, other ad-hoc standards and spin-offs such as package.json `browser` fields and import maps, esm vs cjs vs umd woes... any of those could break and the error messages are impenetrable when they do.
If what you’re doing works via script tag, the odds of Vite/CRA/etc not working for you are extremely low, so why worry about problems you might have before you ever have them. It’s perfectly fine to opaquely trust your tools while focusing on the details you actually care about.
In another context, no Java course ever starts off with explaining “public static void main(String[] args) {…}” and students just “cargo-cult” it for a while until through other exposures to various bits of functionality through natural use they build a context through which to eventually circle back and easily understand what that crucial bit of code actually means. And that’s long before someone would consider trying to get a deeper understanding of what the compiler is doing! “Welcome to CS101, we’re going to start off talking about byte-code and intermediate representations” would have a success rate of literally 0%.
Secondly, here's an analogy: cargo is too much magic, people learning Rust should apply a beginner's mindset by downloading dependency crates themselves and finding the appropriate rustc command lines rather than using cargo build. Sounds unconvincing?
Apart from that, debugging the builds is a bad experience.
For greenfield projects? Certainly. For existing projects? Oh, you're in for a world of pain.
I'm all for breaking with the old and ushering the new era in, but there should be an upgrade path for as many possible combinations of the old as possible.
As luck would have it I decided to convert our project to vite to see if it's feasible.
So far it's:
- Absolute imports don't work and are broken in any number of subtle ways
We use typescript with a setup that resolves `/our-stuff` to `./src`.
Also this is served from `http://our-server/our-stuff` because we're a part of a larger app.
Good luck making this work with vite. Somehow it insists that you should replace all absolute imports with `/@/` and setup an alias for that. All other options (like using vite-tsconfig-paths) will not work because vite's `base` config will break this.
I "solved" it by listing all directories in `src` and converting them into vite aliases.
- Now I'm stuck with `[vite] Internal server error: failed to resolve "extends":"../../tsconfig.json" in <some-path>/mui-theme/tsconfig.json`
How to fix it? Hell if I know.
At this point I'm just giving up, and looking into how to maybe upgrade to webpack 5.
Webpack allows you to paint yourself so far into a corner that it becomes impossible to move anywhere. We should see this for what it is: an existential threat for your app. It eventually grinds everything to a halt.
Which config?
> Webpack allows you to paint yourself so far into a corner
Because vite does, and "just go with vite", right?
But hey, tons of people swear by the simple setup they claim to understand; good for them. As someone who don't work on frontend stuff professionally, I wouldn't pretend I "understand" esbuild and can always fix it when it breaks. Ironically, TFA presents an issue with this simple setup that the author could not figure out without help from experts:
> I was really baffled by this, but I was eventually able to figure it out with some help from people who understood Javascript.
1. Absolute paths are not "bespoke tooling config", it's a tsconfig setting
2. Requiring some code to be served from a specific url is not "bespoke tooling config", it's a requirement for quite a few projects
3. Forcing someone to mindlessly and absolutely needlessly re-write thousands of import statements isn't really a solution
4. This will still not fix "failed to resolve "extends":"../../tsconfig.json"" and who knows what other errors
This is how progress is measured in the web world. The impact of your code/project/patch is measured by how many lines have to be rewritten to adopt it. The higher the number, the better your web wizardry rank is.
> esbuild is not featureful enough for a lot of production use cases
What makes you say that? I keep hearing people say this, but I haven't run into any problem that make esbuild not ready for production.
If you're using any ES6 features - default params, template literals, arrow functions, let/const, spread, etc, your cut-off for browser support will be ~2017 (= no support for IE 11).
swc (https://swc.rs) does transpilation and it's as fast as esbuild. Setup is slightly more complex. I'm hoping esbuild will add ES5 transforms at some point, would much rather have a Go tool underneath.
Regarding #2, I assume that things are set up so that type errors are caught in some other way, such as in CI or in a precommit hook.
My code editor catches 90% of the errors, but it’s definitely nice to have tsc catch the rest earlier than the CI process.
With or without strict mode enabled?
Then when you put in prod, you build once. 30s once a day doesn't matter.
I mean, notepad.exe is arguably a better text editor for most people compared to anything else most of us are using.
I'll happily take some minor overhead in build configuration over adding 30s+ to every automated build if it's not a personal or small-scale community project.
It hardly matters because it's very rare that you need to run build during development. Use dev.
> I keep hearing people say this, but I haven't run into any problem that make esbuild not ready for production.
https://vitejs.dev/guide/why.html#why-not-bundle-with-esbuil...
For the hundredth time: it does matter.
Stop wasting people's time with slow tools and telling them it doesn't matter. That's not a way to treat your users.
Your fancy hot module reloading is not what I want. What I want is to reload the page and see my changes.
I've never seen a "hot reload" that works properly. The one some of co workers are using right now works like this:
- You edit a file
- You wait 5 seconds for the "instant" rebuild
- The browser reloads the page 5 times
- After a total of over 10 seconds, you can finally see the effects of your changes.
Maybe vite's HMR is better, I will never know because I don't care.
The link you sent has the following to answer why not esbuild:
> some of the important features needed for bundling applications are still work in progress - in particular code-splitting and CSS handling.
But this is not true. It handles css and does code splitting. Now, you have to be very explicit about code splitting: you have to define other entry points that share the same libraries as the main program so that it forces those shared libraries into separate chunks. To have more chunks you have more entry points where each entry point uses just one or two libraries. Yes, it's not perfect, but it's not a blocker for getting to production.
Where as the 30 seconds build is a total blocker for development iteration.
And what other people want is to not having to reload everything, lose all state on the page and make API requests all over when they add a 2px margin.
> I've never seen a "hot reload" that works properly.
Yeah, you've never seen a hot reload that works properly means someone else can't develop something new that works. And whatever crappy workflow that you enjoy must be great for everyone else.
Anyway, talking to close minded individuals is pointless, so this is it.
That's the difference.
I don't make API request in local development mode.
If I do, the response is instant.
> Yeah, you've never seen a hot reload that works properly means someone else can't develop something new that works.
It might work, but if the cost is increasing build time from 1 second to 30 seconds, it's not worth it.
The JS ecosystem is riddled with such pests, and (as with webpack) eradication from our workflows always proves by far the best policy.
I’ve been told that about jspm, webpack, parcel, snowpack, babel w/ rollup, browserify, and God only knows how many others.
Don’t you guys get tired of all this churn?
The fact of the matter is, most of these projects came along for justifiable reasons, and represent some improvements over the existing tech: speed, ease of configuration, better dev experience, better use of platform features, and so on. I'll certainly grant that there's an aspect of chasing the new shiny here, but among the senior FE folks I know, the costs of migration are always being weighed against the actual benefits you get.
Additionally, the cost of adoption/migration is very often a high-priority concern by the people writing these libraries. The pendulum is swinging back from configuration to convention.
With regards to the pace: it mainly has to do with the fact that the Web-as-application-platform has only come into its own in the past 10-15 years or so (i.e. since the death of Flash.) The tooling is just now catching up. Another HUGE factor is the growing adoption of TypeScript- for the past several years, TypeScript tooling and existing JS build/bundle systems have gradually adapted to each other, which has driven change. (Trust me: trying to figure out how to set up TypeScript compilation under webpack was confusing and miserable for a long time.)
If you're just glancing into front-end OSS every once in awhile, or if you're not heavily using the features available in these tools, you might not notice or value these improvements. And that's perfectly OK- you can certainly continue to use whatever's comfortable for you. But there's more here than meets the eye.
I'm dealing with a complex webpack/babel setup at work. It is rife with issues, and it amazes me that these are issues that we created ourselves. Endless dev time is wasted on debugging overly complex toolchains.
I think Julia had the right attitude. Instead of taking the leap of faith with a popular tool that you don't understand, instead, choose a smaller, focused tool that you can understand. As you build your application, you can more clearly and organically identify your painpoints, and then you re-evaluate if a more complex tool would be a better fit.
I think going the other way around (as you seem to suggest) is a common mistake made by many entering the JS dev world.
EDIT: For the record, I use Vite, esbuild, and rollup across several projects.
Would you help me set it up to process SCSS with dart-sass and load stimulusjs controllers as well as possibly jquery and some other packages in conjunction with 11ty, middlemanapp or rails?
All I need is processing of scss and mostly vanilla js.
Is this possible?
1. https://jvns.ca/blog/2021/04/03/what-problems-do-people-solv...
It's glorious.
In case it's handy, the two options I find myself telling people about most often are: '-f' to keep tracing across forks, and '-s' to increase the number of characters of strings that get shown when you're trying to debug complete reads/writes.
(filtering for specific syscalls and stuff is also really neat but most of the time I just throw a full strace into a file and chop it up with my usual unixy tools from there)
esbuild is one of the few good tools, it has a ton of features and it's still fast and lean.
Would like to direct to skypack.dev too for zero build tool imports as well!
Based on https://docs.skypack.dev/skypack-cdn/api-reference/pinned-ur..., it appears that you can do a. with Skypack, but this requires a manual step: look up the package in the CDN with curl or your browser and copy-paste the URL into your JS import statement. Is there any tooling to automate this?
Also, there appears to be no way to fail the build if the contents of the pinned URL change. Are Skypack users relying on Skypack to ensure that can't happen?
I've generated and hacked dozens of autotools builds and written m4 macros of my own; I've hacked CMake builds; written RPM spec files; fixed Rcpp package installs; written and modified innumerable Makefiles; setup Tomcat webapps; and use Python for most of my daily work. I've even hacked around with Boost's build thing.
Complexity is not the issue with JavaScript. Stability is the problem.
Most devs do not want to "know" build systems. You want it to work most of the time and when it doesn't, you want to be able to find the answer to a problem. You want that knowledge to accumulate over time so that the answer you found the last time still works.
Autotools is an insane system, but it's been the same insane for 30 years.
Currently using vue-cli with Vue 2 and Vuetify, a configuration depending on a lot of precisely pinned versions to get functional. There is a pile of deprecated library warnings during the lengthy build process that I have no idea how long it will take me to address. Could be a couple of hours, could be several days, could be impossible.
Would like to try Vite or esbuild because God knows we could use the speed up, but after investing a couple of weeks to figure out the precise balance of versions that will work and propagating to our 8 or 9 applications, where are we going to be next year? I saw a Tweet yesterday about vite on swc. Is that going to be the winner?
I half wish Richard Stallman would take over. How desperate is that?
Building JavaScript is just a fucking disaster. I’ve used a fair number of different bundling tools—Browserify, Webpack, Rollup, Esbuild, and Closure (the compiler). I’ve also used old-school concatenation… anything from <script> tags to something like “cat”. It’s just fucking awful. Stay on the straight, narrow path and you’ll survive. Deviate slightly and you’re doomed.
Making things worse, you might have two different environments you run your code in. Node.js and the browser. Surprisingly, the browser lurches ever fowards, and Node.js is the anchor keeping us in the past. Making things worse, you might try to use TypeScript. Making things worse, you might use JSX.
Nearly every JS project I work on needs a ton of babysitting w.r.t. the build system. At various places I’ve worked, there might be a team handling that for me, but if I’m working on a side project, it is very difficult to keep the complexity of the build system down to a reasonable level. Most guides on how to use frameworks will tell you to do something like “oh, just use create-react app” or similar, and you end up with a couple dozen new dependencies. Your build system will be a mix of templated code pasted in to your repo and third-party libraries. Integrating with anything else often requires various "adapter" dependencies, but it's a roll of the dice whether those libraries are built reasonably.
My basic desire is often a fairly simple list… I want front-end TypeScript code to be type-checked and bundled, I want a dev server that serves the bundle, I want to see build errors quickly and easily, I want to run tests without a browser environment. I know this is possible, but every time I’ve gotten it, it’s taken an unreasonable amount of effort.
The thing is, as the same somebody noted, that then it stays trained.
To quote from upthread, "the same insane for 30 years" is in and of itself a huge feature that makes up for quite a lot of insane.
And CMake still seems extremely C/C++ focused. I don't want a language-specific build tool anymore. Heck, I'm not even sure I necessarily need a build tool so much as a generic task runner with patterns and complex/dynamic dependencies.
Things are getting better. but js has definitely had more churn, and reinvented more wheels than anything else i know of.
Re-implemented a complex setup using it in an afternoon with very few hiccups: https://github.com/cheatcode/joystick/blob/master/cli/src/fu....
It's not the usual JS personality screaming "wEbPaCk iS dEaD!!!lol" it's clear the author is thoughtful (observable via the project itself, the docs, and how he interacts with people in Github comments).
Restored my faith quite a bit.
It's still a risk, but in this case one I suspect I'll also take next time I need to add a build step to a JS project.
I don't learn any transferable skills writing Webpack or Babel config. They don't even necessarily transfer to the next generation of the same tool.
I would also take a stable autotools over an unstable autotools, and I consider most of JS-land roughly equal in usability to unstable autotools. Autotools was also not that bad on the happy path where you were targeting, say, the top five POSIX systems over ten years - and maybe you didn't even need it at all, if you didn't need the performance from distinguishing each platform's best supported fd polling variant. Vs. JS, which is still bad even if you're only targeting evergreen browsers, because maybe you're stuck trying to integrate CSS modules with TypeScript or something. That kind of problem simply doesn't exist in autotools world.
It's not that I prefer those tools; they are really obtuse. It's just that I can't handle the moving target of JavaScript. I'm sure I can figure out esbuild just fine, but in a year or two it could be abandoned in favor of swc or Vite or a Vite-esbuild-swc uber package.
I've never touched the old JS guts, `module.exports`,the prototype chain, ... Babel (and later TypeScript) allowed me to live in ESM land from the beginning. I expect any tool/library to be available using exclusively on npm, in reality, I've been constantly fighting with it over the past few years.
I usually get a shock when I see libraries still offering minified bundles that assign to the global namespace, to me this seems like outdated practice, but I guess that this is still really prevalent in JS, even in 2021. I sometimes want to go back to it, with the amount of configuration and boilerplate necessary to set up a project with npm, Prettier, ESLint, Typescript, Webpack/Parcel/Rollup, SPA vs MPA, React vs Vue vs $SPAFRAMEWORK, it was a lot easier back in the day.
I think the biggest failure in the JS system is the lack of standardisation, it's so versatile that almost anything is possible. A number of different module systems were invented to solve the initial growing pains (the first would literaly replace a `require()` function call with the contents of the given file), and we never escaped them as they became the foundations of Node.js/npm. Then we needed a non-blocking variant, so we just called it `import()`, it returns a promise. Then we wanted proper imports, so we invented the `import ... from ...;` ESM syntax. We made a named import and default import variant (with most bundlers offering a compatibility layer with CJS-type exports, so imports now have a hidden `__esModule: true` key...). In order to stay compatible with legacy CJS, we also added the `import * as ... from ...;` syntax as an escape hatch?
When TypeScript came along, in order to be compatible with JS it became common practice to distribute compiled TS code along with the source code, with an adjacent declaration `.d.ts` file to provide type information. The source code was never leveraged in any way, and so people just stopped shipping the `src` directory, only `dist` and a `package.json`. It makes debugging a pain, but compiling TS libraries would have been tricky, considering the widely varying different feature flags (async, DOM, ...) and strictness options. Perhaps Deno will help with this to some regard. The great thing about Go or Rust (or any other language, really) is that you import source code, not illegible compiled code (unless you're dynamically linking). I believe that Rust is especially good in this regard, it remains compatible with older code by requiring crates to specify a Rust "edition".
Why are we in this mess in this first place?
And why has it remained so darn popular?