You may not need a bundler for your NPM library

i think bundling generates your build from the source, but i mean generating a source from a build. so what i mean is more like reverse bundling.

so that you don't have to pull node_modules every time and install. i;m not totally sure but i dont think you can turn a minified bundled js files back into their original node_module source files...

> Hmm, why don't you specify your development-time dependencies as devDependencies in your package.json? Then you can simply install only the production ones in your final step Docker image - and only use the devDependencies in your build step.

This is pretty close to the solution and would work if I ran just Node.js server side. Then I could easily have separate sets of dependencies for development and deployment/running, which is what most folks should do. Yet in certain tech stacks, things aren't as easy and there is reliance on system folders instead of local ones.

For example, my application that's running on the server should have absolutely nothing to do with Node, since it should run Ruby alone and use the bundled files, however it should have all of the Ruby Gems (basically packages) available. Of course, I also need Node and possibly something else during build time, for generating my assets.

For example, my intermediate container images might look like:

  FROM ruby_with_node_and_python AS builder

  # Copy only what's needed for installing dependencies and other garbage (e.g. changed code files won't invalidate Docker cache)
  COPY ./src/.browserslistrc ./src/.ruby-version ./src/babel.config.js ./src/config.ru ./src/Gemfile ./src/Gemfile.lock ./src/yarn.lock ./src/package.json ./src/postcss.config.js ./src/Rakefile /app

  # install front end dependencies
  RUN yarn install
  # add code and run asset creation
  COPY ./src /app
  RUN bundle exec rake assets:clobber RAILS_ENV=production
  RUN bundle exec rake assets:precompile RAILS_ENV=production

  # install back end dependencies
  RUN bundle install

  # run tests (local SQLite, for example)
  RUN rails db:migrate RAILS_ENV=test
  RUN rails test RAILS_ENV=test

  # clean up files after build (so copying in later images doesn't drag garbage along; this could be a separate intermediate image too)
  RUN rm -rf src/node_modules
  # clean up files after test (so copying in later images doesn't drag garbage along; this could be a separate intermediate image too)
  RUN rm -rf src/log/*.log src/tmp/*

And then my container images for deployment would look like this:

  FROM ruby_with_nothing_else AS runner

  # copy over application files
  COPY --from=builder /app /app

Except that this doesn't work! Why? Because Ruby Gems aren't installed in the application directory, but instead sit in a variety of directories on the actual file system. So instead I'd need something like:

  FROM ruby_with_package_manager AS runner

  # we actually could not carry over the precompiled gems etc. because I'm stupid, so instead we now run the install again :(
  # well it's not like you need front end files here, but this doesn't really mean much
  COPY ./src/.browserslistrc ./src/.ruby-version ./src/babel.config.js ./src/config.ru ./src/Gemfile ./src/Gemfile.lock ./src/yarn.lock ./src/package.json ./src/postcss.config.js ./src/Rakefile /app

  # install back end dependencies
  RUN bundle install

  # copy over application files
  COPY --from=builder /app /app

So while npm/yarn allows you to avoid some of the issues with `npm install --production` to exclude the devDependencies, other tech stacks aren't quite there yet. In addition, it feels kind of crazy to ship package managers like npm/yarn/pip/composer/bundle/maven or anything else in containers that should be immutable, so in practice you'd end up with more stages, even if you knew how to carry over the packages from one image to another in the technology stacks where it's not entirely clear how to do that.

I actually tried something similar to this, but it simply didn't work, especially when native extensions were needed for some of the Gems: https://stackoverflow.com/a/33778136

Wait a few days for what, exactly? For your sub-sub-sub dependency to patch it, then for your sub-sub dependency to notice this and update the version, then for your sub-dependency to notice it to update it, then for your dependency to finally notice and update it?

The solution that npm presents solves this problem without this insanity -- you can just create a `npm-shrinkwrap.json` file to manually update sub-dependencies. If one of your dependencies used bundling, this solution becomes impossible.

> certain libraries that don’t work with esm yet

I assume that you're talking about tools rather than libraries. Any simple library can import ESM asynchronously and any ESM packages can import CJS packages even with static imports.

Tools and Node’s strict ERR_REQUIRE_ESM have been the problem with the migration.