obensource

I remember when the ‘tiny module’ revolution (a revitalization of the Unix philosophy) came en vogue in JavaScript circles while Node.js + npm were becoming popular for the first time.

This revolution was possible because the community received and adopted CommonJS (aka CJS) as the single way to bundle modules together outside of the browser. It was a powerful feeling to be able to ‘require’ any library anywhere in a browser-decoupled JS runtime like node, get it all wrapped together in a giant IIFE by Browserify – and then immediately be on your way to running any script, tool, or app you want, on any system or browser runtime, with just JavaScript.

Fast forward to the early 2020s, and the tiny-module driven JavaScript community has seen severe supply chain issues, migration vulnerabilities – and is currently experiencing something between a watershed moment, and a Python 2 vs 3 type of schism over adopting ECMAScript Modules (ES modules, ESM) as its default module system. Since ESM was driven into standardization by TC39, the WHATWG, and some of the JS community (while not receiving support from others), it remains both a permanent way of how we're going to be building JavaScript, and something that many developers will continue to script their way around.

On the surface it may seem like this change mostly introduces a new syntactic sugar into node and across the JS landscape, but the truth is that under the hood: the Web ecosystem has been maturing towards both asynchronicity, and the interoperability between running JavaScript in and out of the browser – to the point that it’s become necessary to revise the way we build and bundle our libraries so they can support this evolutionary stage, and lessen the division between the two arenas.

The Playground is Changing

At this juncture for JavaScript, CommonJS has been a magical one-trick pony that's holding on like a frog in hot water. This environment really began to heat up when differentiating between .cjs, .js, and .mjs file extensions became a runtime requirement, given that they began supporting the (recently specified) way JS module loaders differentiate between CommonJS-adherent, and ESM-adherent JavaScript modules:

.cjs: a CommonJS module designation.

.js: Interpreted as an ES module.

.mjs: an ES module designation.

This is central to the package resolver algorithm that was implemented in both node and browsers. These loaders check to see if the script URL that was passed in ended in .cjs, .js, .mjs, etc – as is now described here in the node spec:

This new requirement of designating more extensions other than .js brought the hard division between these module system options to the surface – adding more complexity to an area that had previously been entirely simple.

One of the most significant factors supporting the ongoing decision to slowly, and fundamentally change how JS module systems work (with some of the more impactful milestones going back to ES6 when import and export statements were added to JS) is that, at its root: CommonJS modules come together synchronously. The module system loads, instantiates, and evaluates them as an untouchable set of enumerated operations before they’re funneled together into a single bundle whenever a build script runs – which is then loaded by the runtime environment for execution, and that’s that. That’s what you’ve got to work with, and there’s no flexibility within the module system’s steps (no callbacks or promises), or around the order they execute, and it’s not something that the browser can tap into.

ES module systems however, expose these operations (loading, instantiating, evaluating) to any JS runtime that supports the spec, and each can be independently run by a loader as needed (like the one you use via HTML when adding a<script> tag).

For the Web, ESM support is a long-term win since module loading isn’t tied to just node anymore. Letting the browser use and manipulate the module system itself has definitely thinned the line for compatibility between the independent JS runtimes (node, Deno, etc) and every common browser. The community knows that using a default module system that isn’t natively interoperable between the two isn’t ideal (CommonJS modules have always had to be transpiled and bundled first before being loaded into the DOM), no matter how painful it may be to migrate to a better solution.

Now that just about every browser supports ES modules, loading an external module (independently of bundle files) is as easy as pointing a <script> tag at it, as long as you tell the script loader it’s an ES module by adding thetype="module" attribute.

If you’re new to ESM, it’s worth mentioning that the system transpiles JavaScript differently than CommonJS: it uses ‘strict’ mode by default, and has different scoping rules. For example, this isn’t a reference to the global object anymore. All of this breaks backwards compatibility for Web apps built with CommonJS, and earlier versions of node.

Warning Signs on the Road Ahead

The road forward to ESM has been long, tedious, and fragmented since module bundling isn’t a single solution that’s tied to just node, and there are many more opinions driving its future now.

What we can count on though, is that we’re going to be using import and export statements for the foreseeable future – and if you’re still supporting or building a Web app with CommonJS your time is running out. You’re going to hit a wall soon that you just won’t be able to get over because CommonJS is continually losing support by popular and commonly used JS libraries (for example: D3, Victory, etc).

My progress was halted at the juncture of using location visualization libraries which rely on D3 for an app I’ve been building for a while now. The state of my dependencies were so interwoven that I couldn’t just declare module bankruptcy and start fresh (which would’ve taken an unnecessarily exorbitant amount of time that I don’t have given the size of the project).

No matter what your hot take is about the migration away from CommonJS (there’s a whole lot of them), it’s now something that must be done if you want to build a new app with modern JS libraries, or to evolve your old one.

When it comes Time to Migrate

There’s an onslaught of new bundlers currently available for ESM, but not all seem to be equal in terms of making it simple to migrate an old app from a CommonJS bundler like Browserify to an ESM bundler like rollup, Parcel, Bun, Vite, Snowpack, or esbuild.

Resources for building a new app with these bundlers are abundant, but there’s not a lot to draw from regarding the experience of a migration between CommonJS and ESM. Because of this, my migration story is mostly one of swapping module systems in and out of my app in the dark, while trying to find a configuration that worked and wasn’t brittle. From here on out I’ll cover my most significant discoveries, complexity pitfalls, and how you might get a bundler migration over the finish line more easily than I did.

Surveying the Ecosystem

Currently, there’s not a lot of guidance around what to consider when selecting a bundler for an ESM migration.

When there’s no clear path, I typically poll opinions from both my immediate colleagues*, and through as large a swath of experienced developers as I can get at the watercooler of twitter.

With the bundler leads I generated by asking around, I proceeded to try rollup (a mature project), Parcel (well loved, with a promise of ‘no configuration’), and then esbuild (widely used, and strongly endorsed) – in that order.

Initial Preparations

There’s a handful of preparatory steps that need to be addressed before you try to uninstall the CJS bundler package, and then attempt to install & configure the ESM one.

In short (I'll expand in detail momentarily), you'll need to:

▹ Tell your linter you’re moving over to ESM so you’re not continually annoyed with unnecessary error messages from the get go.

▹ Update your package.json so that npm knows this is an ESM project.

▹ Update any bundle imports in your index.html file (and/or any subsequent HTML files that use it) so the browser knows that it’s loading ESM modules.

▹ Change all of your scripts (in node) from using require& module.exports statements, to import & export statements.

▹ Address any immediately obvious scoping issues that your linter identifies.

▹ Address any immediately obvious scoping issues that your linter identifies.

Changing how your scripts import and export modules can really be done at any time, I just prefer to do it before installing the new bundler just in case there’s a ‘just works’ moment (which happens very rarely, but they’re definitely fun when they do).

Now let’s break this down into something a little more granular and helpful.

Linting

To support ESM, your linter will need to know that it’s parsing it. You can do that by declaring the source type to be ‘module’ in your config (so it becomes enabled). If you’re using ESLint, it’s going to look something like this in your .eslintrc.json (or similar for whatever format you use for eslint):

The only caveats to remember with ESLint here are that it doesn’t support top-level await unless you’re using version 8 or greater, and you’ll need to enable the latest ECMA features as well (for example: "ecmaVersion": 13).

Updating your package.json

Your package manager is going to need to know about the type of modules it’s working with, and in order to do that all you need to do is declare the project to be of "type": "module" as a first-level property in your package.json object.

HTML Imports

Now you can let your Browser know what it’s trying to load. This is only a slight modification to the scripts you’ve already been loading in your index.html file (and to any other HTML files that use them).

Just add type="module" to any existing script tags (and if you’re using the .mjs extension, you’ll need to serve those files using the correct MIME type), and then you’re on your way.

Changing how you import & export your scripts in node

Warning: you’ve hit the migration checkpoint of tedium and linter complaints. It’s time to change every require() declaration in your code to an import, and every module.exportsdeclaration to an export. It’ll feel better over time as your linter yells at you less and less though, just hang in there.

Address Scoping Issues

Remember how I said that ESM fundamentally changes the scope of your JavaScript, global variables, etc? Well, node will make that immediately apparent for you. You’ll see right away that node’s global objects (for example: process, and __dirname) aren’t recognized anymore. There are a number of ways to address this, but the easiest route towards just getting the migration done will likely be to import whichever node core module you need to accomplish the same task (like getting the local dirname).

Uninstalling the CJS Bundler

Congratulations! You’ve made it to the last step of preparation before you turn on the ESM firehose.

It’s time to uninstall your CJS bundler, and move on.

In my case, I’ve used Browserify – so it looks like this:

npm uninstall browserify

Note: If you’re running npm version 7 (or earlier) you’ll need to add a -D or --save-devflag to the uninstall command, so browserify and other unneeded dev dependencies are removed from your project.

Fortunately, Browserify doesn’t really leave any artifacts lying around other than the bundle script it generated, so you can delete that too before moving along if you’d like.

Installing and configuring your new ESM bundler

Now we get to the heart of what we want to accomplish here.

First, let’s recall how this was done before we change it up. Here was the Browserify way:

browserify -t brfs src/public/js/app.js > src/public/js/bundle.js

The -t and brfs flags invoke a transform module on the top-level file that applies some pre-import transformations on it so that it’s easier to load static assets into the bundle, and more easy to transpile the source into another flavor of JavaScript, or language. If you’re interested in how these work, you can read more about that here.

Now the mission ahead is to choose a bundler, replace this bundling command that lives in your package.json, and then properly configure it (which will differ with each bundler since they’re all implemented differently).

For my migration, I first reached for rollup.js.

Rollup.js

After installation, the first move to make was to rewrite my npm buildscript in the package.json. This is where I pointed rollup at my app’s top-level source file, specified where the bundle needs to go, and then how the bundle should be formatted:

"build": "rollup src/public/js/app.js --file src/public/js/bundle.js --format iife"

Browserify transformed the entire bundle into one big giant IIFE, so instructing the bundler to use that same format seemed preferable.

In my case, I had gone against the better judgment that I outlined above of just completely migrating to ESM import and export statements first, thinking that I could temporarily get by with a config that transpiles my ESM modules back to CJS, but this seemed to be a brittle option and led me to a broken front end because the modules weren’t properly loading.

I then threw my hands up with that (this brittle duct tape fix did not deserve any more time), and just followed through with the full syntax migration.

Then I began to see this external dependency issue:

Given that rollup was nice enough to point me to their documentation regarding this, I came to discover something hair-bendingly awkward about this module system: you are required to manually specify external dependencies in your bundler config, or else they won’t be included in your bundle.

I then attempted to manifest the precise configuration (incantation, rather) in my rollup.config.js file that I might need to get this working.

Specifying the correct properties here proved to be unintuitive. For example: I assumed that adding module names to an external property would let rollup know that it needs to include those modules in the bundle. However, rollup behaves in the exact opposite manner – and the purpose of that property is to exclude any named modules from the bundle. So what I had actually done here was to instruct rollup to attach these modules to the window object, and then exclude them from the bundle (hence, the ‘missing global names’ and ‘guessing…’ errors seen earlier).

I then removed the external property, assuming that leaving the modules specified in the globals property might bring it together since the purpose of that field is to attach them to the global window. Still though, I was left with unresolved dependencies and global scope issues.

My last ditch effort was to reference a colleague’s battle-tested rollup config, but no additions or reductions made any real difference.

At this point, I realized this bundler is just not the way to go if I’m going to be building for the Web – and after getting a headache from my experience with its very config heavy requirements, I then bought into the promise of the toutedly ‘0 configuration bundler’ (boldly declared on their site): Parcel.

Parcel

Parcel came with the expectation that it might ‘just work’, and it seems to be a favorite among front end devs at the moment – but these wonderful endorsements come from individuals building projects that started fresh with Parcel, rather than having performed a migration to it.

There’s an old sentiment among Web developers that goes something like this: The more magic you get up front, the less you’ll be able to know what’s going on, and it’ll destroy your ability to fix anything.

This is what I found to be true about Parcel in the case of a migration. Running this bundler is definitely simple enough (parcel build), and it has very beautifully descriptive and accurate CLI error messages – but it abstracts away a little too much logging from the runtime to know what it’s trying to do with the input and output files.

Seeing that my UI was loading, but the modules weren’t, I soon found out that when migrating: the ‘zero configuration’ bundler needed some basic configuration.

Parcel allows you to specify build targets for both the front end and back end – so I tried being explicit with that.

Unfortunately, this didn’t resolve my issue, and I couldn’t really tell why because by all accounts it seemed like the bundler was configured correctly – but it was swallowing errors before they made it to the browser.

Tired of feeling the magic that didn’t really get me to where I wanted to be, I decided to move on again to a new bundler.

As I was removing Parcel from my project though, I noticed a very populated new directory called .parcel-cache (an artifact from Parcel’s way of making rebuild times faster).

The next tool I reached for was recommended by some of my closest colleagues, who say they use it exclusively now: esbuild.

ESBuild

esbuild is a Web-first bundler, so I came in with high hopes for this one.

After installation, I added the build script to my package.json:

esbuild src/public/js/app.js --bundle --outfile=src/public/js/bundle.js

This loaded the modules, but came back with a scoping error for the global object being undefined. My errors seemed to be more verbose and helpful with esbuild since it provides more native browser support.

Given the error, I quickly found out that I could specify the global object directly in my build script.

With that, it worked! Now there was no perceivable difference between this, and the way I had previously built the app – and I was on my way to using ESM-only tools like D3.

Being that esbuild is a minimal configuration builder that’s made with building for the browser environment in mind – it fit beautifully for a migration to ESM like this. It operates similarly to how Browserify does, and allows you to just ‘set it and forget it’.

The bundler survey results are in

Being that esbuild checks all the boxes for a clean ESM migration if you’re: building a Web app, coming from CJS, and want to continue using major libraries in the JS ecosystem – I hope you reach for this bundler first.

* Special thanks to Benjamin Koltes, Bret Comnes, and Bryan English for deepening my understanding of: ESM, the current landscape of JS bundling, and how a few popular ESM build tools actually work.