Skip to content

Instantly share code, notes, and snippets.

@joepie91
Last active May 9, 2025 15:29
Show Gist options
  • Save joepie91/bca2fda868c1e8b2c2caf76af7dfcad3 to your computer and use it in GitHub Desktop.
Save joepie91/bca2fda868c1e8b2c2caf76af7dfcad3 to your computer and use it in GitHub Desktop.
ES Modules are terrible, actually

ES Modules are terrible, actually

This post was adapted from an earlier Twitter thread.

It's incredible how many collective developer hours have been wasted on pushing through the turd that is ES Modules (often mistakenly called "ES6 Modules"). Causing a big ecosystem divide and massive tooling support issues, for... well, no reason, really. There are no actual advantages to it. At all.

It looks shiny and new and some libraries use it in their documentation without any explanation, so people assume that it's the new thing that must be used. And then I end up having to explain to them why, unlike CommonJS, it doesn't actually work everywhere yet, and may never do so. For example, you can't import ESM modules from a CommonJS file! (Update: I've released a module that works around this issue.)

And then there's Rollup, which apparently requires ESM to be used, at least to get things like treeshaking. Which then makes people believe that treeshaking is not possible with CommonJS modules. Well, it is - Rollup just chose not to support it.

And then there's Babel, which tried to transpile import/export to require/module.exports, sidestepping the ongoing effort of standardizing the module semantics for ESM, causing broken imports and require("foo").default nonsense and spec design issues all over the place.

And then people go "but you can use ESM in browsers without a build step!", apparently not realizing that that is an utterly useless feature because loading a full dependency tree over the network would be unreasonably and unavoidably slow - you'd need as many roundtrips as there are levels of depth in your dependency tree - and so you need some kind of build step anyway, eliminating this entire supposed benefit.

And then people go "well you can statically analyze it better!", apparently not realizing that ESM doesn't actually change any of the JS semantics other than the import/export syntax, and that the import/export statements are equally analyzable as top-level require/module.exports.

"But in CommonJS you can use those elsewhere too, and that breaks static analyzers!", I hear you say. Well, yes, absolutely. But that is inherent in dynamic imports, which by the way, ESM also supports with its dynamic import() syntax. So it doesn't solve that either! Any static analyzer still needs to deal with the case of dynamic imports somehow - it's just rearranging deck chairs on the Titanic.

And then, people go "but now we at least have a standard module system!", apparently not realizing that CommonJS was literally that, the result of an attempt to standardize the various competing module systems in JS. Which, against all odds, actually succeeded!

... and then promptly got destroyed by ESM, which reintroduced a split and all sorts of incompatibility in the ecosystem, rather than just importing some updated variant of CommonJS into the language specification, which would have sidestepped almost all of these issues.

And while the initial CommonJS standardization effort succeeded due to none of the competing module systems being in particularly widespread use yet, CommonJS is so ubiquitous in Javascript-land nowadays that it will never fully go away. Which means that runtimes will forever have to keep supporting two module systems, and developers will forever be paying the cost of the interoperability issues between them.

But it's the future!

Is it really? The vast majority of people who believe they're currently using ESM, aren't even actually doing so - they're feeding their entire codebase through Babel, which deftly converts all of those snazzy import and export statements back into CommonJS syntax. Which works. So what's the point of the new module system again, if it all works with CommonJS anyway?

And it gets worse; import and export are designed as special-cased statements. Aside from the obvious problem of needing to learn a special syntax (which doesn't quite work like object destructuring) instead of reusing core language concepts, this is also a downgrade from CommonJS' require, which is a first-class expression due to just being a function call.

That might sound irrelevant on the face of it, but it has very real consequences. For example, the following pattern is simply not possible with ESM:

const someInitializedModule = require("module-name")(someOptions);

Or how about this one? Also no longer possible:

const app = express();
// ...
app.use("/users", require("./routers/users"));

Having language features available as a first-class expression is one of the most desirable properties in language design; yet for some completely unclear reason, ESM proponents decided to remove that property. There's just no way anymore to directly combine an import statement with some other JS syntax, whether or not the module path is statically specified.

The only way around this is with await import, which would break the supposed static analyzer benefits, only work in async contexts, and even then require weird hacks with parentheses to make it work correctly.

It also means that you now need to make a choice: do you want to be able to use ESM-only dependencies, or do you want to have access to patterns like the above that help you keep your codebase maintainable? ESM or maintainability, your choice!

So, congratulations, ESM proponents. You've destroyed a successful userland specification, wasted many (hundreds of?) thousands of hours of collective developer time, many hours of my own personal unpaid time trying to support people with the fallout, and created ecosystem fragmentation that will never go away, in exchange for... fuck all.

This is a disaster, and the only remaining way I see to fix it is to stop trying to make ESM happen, and deprecate it in favour of some variant of CommonJS modules being absorbed into the spec. It's not too late yet; but at some point it will be.

@guest271314
Copy link

Yes. Manually, and programmatically. I have briefly tested JerryScript recently. I was looking at boa yesterday.

@Kreijstal
Copy link

Yes. Manually, and programmatically. I have briefly tested JerryScript recently. I was looking at boa yesterday.

I mean obviously if you want to support all runtimes the idea is transpilation, no need to split into files, have everything in one big file, then convert everything to ES3, no?
But here it's about CJS, how it doesn't work on the browsers like ESM.

I can always (import("https://esm.sh/canvas-confetti@1.6.0").then(function(confetti){confetti.default()})) on a site and it will work

@guest271314
Copy link

I've done it a few different ways. Sometimes manual, line by line substitution, writing is necessary.

Here's reading stdin to node, deno, and bun using the same script https://github.com/guest271314/NativeMessagingHosts/blob/main/nm_host.js

/*
#!/usr/bin/env -S /home/user/bin/deno -A /home/user/bin/nm_host.js
#!/usr/bin/env -S /home/user/bin/node /home/user/bin/nm_host.js
#!/usr/bin/env -S /home/user/bin/bun run /home/user/bin/nm_host.js
*/
import * as process from "node:process";
const runtime = navigator.userAgent;
const buffer = new ArrayBuffer(0, { maxByteLength: 1024 ** 2 });
const view = new DataView(buffer);
const encoder = new TextEncoder();
// const { dirname, filename, url } = import.meta;

let readable, writable, exit; // args

if (runtime.startsWith("Deno")) {
  ({ readable } = Deno.stdin);
  ({ writable } = Deno.stdout);
  ({ exit } = Deno);
  // ({ args } = Deno);
}

if (runtime.startsWith("Node")) {
  readable = process.stdin;
  writable = new WritableStream({
    write(value) {
       process.stdout.write(value);
    }
  });
  ({ exit } = process);
  // ({ argv: args } = process);
}

if (runtime.startsWith("Bun")) {
  readable = Bun.file("/dev/stdin").stream();
  writable = new WritableStream({
    async write(value) {
      await Bun.write(Bun.stdout, value);
    },
  }, new CountQueuingStrategy({ highWaterMark: Infinity }));
  ({ exit } = process);
  // ({ argv: args } = Bun);
}

function encodeMessage(message) {
  return encoder.encode(JSON.stringify(message));
}

async function* getMessage() {
  let messageLength = 0;
  let readOffset = 0;
  for await (let message of readable) {
    if (buffer.byteLength === 0 && messageLength === 0) {
      buffer.resize(4);
      for (let i = 0; i < 4; i++) {
        view.setUint8(i, message[i]);
      }
      messageLength = view.getUint32(0, true);
      message = message.subarray(4);
      buffer.resize(0);
    }
    buffer.resize(buffer.byteLength + message.length);
    for (let i = 0; i < message.length; i++, readOffset++) {
      view.setUint8(readOffset, message[i]);
    }
    if (buffer.byteLength === messageLength) {
      yield new Uint8Array(buffer);
      messageLength = 0;
      readOffset = 0;
      buffer.resize(0);
    }
  }
}

async function sendMessage(message) {
  await new Blob([
    new Uint8Array(new Uint32Array([message.length]).buffer),
    message,
  ])
    .stream()
    .pipeTo(writable, { preventClose: true });
}

try {
  // await sendMessage(encodeMessage([{ dirname, filename, url }, ...args]));
  for await (const message of getMessage()) {
    await sendMessage(message);
  }
} catch (e) {
  sendMessage(encodeMessage(e.message));
  exit();
}

export {
  encodeMessage,
  exit,
  getMessage,
  readable,
  sendMessage,
  writable,
};

A while ago I bundled ts-ebml for the browser. There is a polyfill for require() https://github.com/legokichi/ts-ebml/blob/a7ec9ddd54c2ff7ae7c0d3656864d0388eda238b/lib/ts-ebml-min.js#L4C1-L5C46

// see requiring npm modules in the browser console <https://gist.github.com/mathisonian/c325dbe02ea4d6880c4e>
// usgae `const tsebml = require("ts-ebml");`

Bun compiles require() into ECMA-262 Modules to an appreciable degree. I don't think a 1:1 conversion is possible; see evanw/esbuild#3947. Somewhere in esbuild repository there's a sentence in an issue where the maintainer says something like conversion between CommonJS and ECMAScript Modules 1:1 is impossible.

It was far more line by line modification to adjust code that was written exclusively for Node.js API's, for example substituting Web Cryptography API for node:crypto - which cannot be polyfilled (see https://github.com/guest271314/webbundle, https://github.com/guest271314/wbn-sign-webcrypto), than dealing just with the module loader, here Build/rebuild wbn-bundle.js from webbundle-plugins/packages/rollup-plugin-webbundle/src/index.ts with bun.

For implementing WASI for Deno, Node.js, Bun, I had to remove the maintainer's Deno references, and use Node.js API's https://github.com/guest271314/deno-wasi.

Notice the different approaches, basically the opposite, between the two examples above.

See using bun to get npm to compile to a standalone executable using node's SEA, along with adding some require("module") in the head of the script https://gist.github.com/guest271314/c9543a19d8ccf72881355b27d0107551.

node -e 'const file="bun-npm-bundle.js";const fs=require("fs");fs.writeFileSync(file, "require=require(\"node:module\").createRequire(__filename);\n" + fs.readFileSync(file, "utf8").replace("#!/usr/bin/env node", ""));'

It depends. In general, manual modification even of code bundled with a bundler might be necessary. There's no roadmap for the experiments and tests I do. JavaScript programmers, from my observation, tend to develop preferences and brand loyalties, and stay in Node.js world. There are exceptions. Some Node.js contributors/maintainers/members, whatever have contributed to QuickJS NG and Deno, etc.

I suggest testing bun build or esbuild, or as you mention esm.sh if the idea to convert CommonJS to ECMAScript Modules.

@Kreijstal
Copy link

what do you mean crypto can't be polyfilled? It won't be constant time, but it can be polyfilled.

@guest271314
Copy link

@Kreijstal

Can't be polyfilled.

[AskJS] Why is the internal crypto module so difficult to bundle as a standalone portable script?

The crypto module is not entirely Javascript. Some of its implementation uses native code which only runs in the nodejs environment and some of its implementation uses nodejs internals (like the thread pool).

Use Web Cryptography API for wbn-sign-webcrypto, crypto-browserify throws for subtle, doesn't implement Ed25519 #786

@Kreijstal
Copy link

Kreijstal commented Jan 13, 2025

Can't be polyfilled.

just because implementation is lacking doesn't mean it can not be polyfilled.
https://github.com/paulmillr/noble-ed25519
it is not an universal truth, maybe it can't right now, because there is no implementation written. Not that it can't be done, can't be written.

@guest271314
Copy link

I said node:crypto can't be polyfilled. We have standardized Web Cryptography API. We have standardized ECMAScript Modules. Node.js is still using non-standard CommonJS. Now, does that mean implementers and programmers in the field should always follow specifications? No. WHATWG Fetch specification, the last time I checked, still has Request resolving to a Promise only when the body has been completely read. HTTP allows for response while the request is still being read. That is, full-duplex streaming using fetch() whatwg/fetch#1254. Node.js, Deno, and now Bun oven-sh/bun#7206 support full-duplex streaming with fetch(), as demonstrated here https://github.com/guest271314/native-messaging-nodejs/tree/full-duplex, here https://github.com/guest271314/native-messaging-deno/tree/fetch-duplex, and here https://github.com/oven-sh/bun/files/13400754/full_duplex_fetch_test.js.zip, although WHATWG Fetch doesn't spell out the mechanics.

@Kreijstal
Copy link

Kreijstal commented Jan 14, 2025

I said node:crypto can't be polyfilled.

What methods or funtions of node:crypto can't be polyfilled?

@guest271314
Copy link

This is the short answer

[AskJS] Why is the internal crypto module so difficult to bundle as a standalone portable script?

The crypto module is not entirely Javascript. Some of its implementation uses native code which only runs in the nodejs environment and some of its implementation uses nodejs internals (like the thread pool).

The long answer is I will have to go back over all of the questions I asked in different venues, and the code I ran in Deno and Bun when trying to write runtime agnostic JavaScript that just so happened to use node:crypto module, because the maintainers of wbn:sign decided to write code exclusively for Node.js. The path to the short answer included getting banned from a couple of repositories. But I finally figured out how to write the code without using node:crypto.

A brief summary, not in any particular order

But if you think my claim is erroneous, all you have to do is create a single script that runs the same in node, deno, and bun starting with this code

Here's my fork

node:crypto is like node:wasi that depends on uvwasi internally.

@Kreijstal
Copy link

But if you think my claim is erroneous, all you have to do is create a single script that runs the same in node, deno, and bun starting with this code

You want to connect to do TCP/IP from node, deno and bun? Also the browser, or just those 3? Do you need crypto for bare TCP/IP?

Right now I'm using node nightly, deno canary, bun canary, hermes, shermes, workerd, d8 (V8 shell), js (SpiderMonkey shell), llrt, qjs, tjs.

I discovered a runtime for you https://duktape.org/ also look at the page of Similar engines

@guest271314
Copy link

I don't need node:crypto, The authors of https://github.com/GoogleChromeLabs/telnet-client decided to write the code exclusively for Node.js. I expanded covereage to Deno and Bun, too, in part by substituting Web Cryptography API for nod:crypto after I found out node:crypto can't be polyfilled.

Yes, I've read about duktape. A list of JavaScript engines, runtimes, interpreters.

@pcj
Copy link

pcj commented Feb 24, 2025

The number of hours I've wasted on ESM vs CJS... And here I am again in 2025 trying to make jest work with ESM, for the nth time. Fuck me.

@Kreijstal
Copy link

The number of hours I've wasted on ESM vs CJS... And here I am again in 2025 trying to make jest work with ESM, for the nth time. Fuck me.

I use tape works everywhere

@Rush
Copy link

Rush commented Feb 24, 2025

The number of hours I've wasted on ESM vs CJS... And here I am again in 2025 trying to make jest work with ESM, for the nth time. Fuck me.

I am just glad to see the rant is alive. We are happily using CJS and staying productive. We use webpack to bundle both server and client code and mostly don't cure about all of this bullshit. Bundled code is all .cjs.

@guest271314
Copy link

Reads like corporate users of exclusively Node.js from 10 years ago.

It's great programmers have choices.

Will the next generation who fill current CommonJS users shoes in companies have the same commitment to CommonJS in 10 years?

Unlikely.

As unlikely as somebody predicting the creator of Node.js would create Deno that uses ECMAScript Modules, though still supports .cjs.

By the way, there's a whole world of JavaScript outside of Node.js itself: from QuickJS to Static Hermes to AssemblyScript to Bun which can run WASM diectly, C via TinyCC, bundle,and support .cjs and ECMAScript Modules, which in general the current and next genereation of JavaScript programmers use.

There's enough room for everybody and their philosophocal and technical preferences in JavaScript world.

@ericchase
Copy link

yes, i think esm has some serious flaws. it seems like it was made from a very one track perspective. maybe for monolithic projects? has anyone found some other places where conversation is happening around this? seeing one guy flood the convo with "you can do whatever you want" isn't exactly useful.

@mk-pmb
Copy link

mk-pmb commented Apr 6, 2025

Sooner or later I'll have to find a good solution, so feel free to subscribe that thread to get notified.

@Rush
Copy link

Rush commented Apr 7, 2025

Just bundle everything and then .cjs and .mjs doesn't matter :-)

@mk-pmb
Copy link

mk-pmb commented Apr 7, 2025

In a not-website scenario, the downside of bundling everything is the waste of disk space and/or transmission time. It also complicates the effort required for updating popular dependencies used in many of your programs. It's why mankind invented shared libraries.
However, thinking about it, maybe using a package manager with a transpile step inserted after each install, may be a worthy stopgap. Not sure how feasible it is though. I hope the alternatives mentioned above will turn out to be even better, and easier.

@guest271314
Copy link

I'm just not seeing a problem statement anywhere here. It's 2025. Bundling everything to a standalone script, or just ECMAScript Modules in individuals files is not only technically possible, that capability is shipped with deno and bun. The only folks who havn't caught on is folks using node version from the past, instead of just using nightly release, consistently.

@iambumblehead
Copy link

Language users benefit from stable churn-free foundations. Most users don't want rely on third-party bundlers to support math, logic or string operations either.

@guest271314
Copy link

stable churn-free foundations

No such state exists, anywhere, in any domain of human activity. If things were "stable churn-free" Moore's Law wouldn't be true and correct. People wouldn't be rollingaround with Android or Apple devices. They'd still be using the first Kyocera "smart phone", Palm Pilot, Blackberry. I guess you could break out a Commodore 64, run Node 20 though nightly is on 24, and put that mobile device down, and read a book, by yourself, with all electronics turned off. That's close to stable.

Other than that, technology is not static, and not fair. If technology was fair Beta would have lasted longer than VHS. CD's wouldn't have cost less than a dollar and sold for upwards of 20 dollars - before CD's essentially became obsolete. Ask Tower Records.

@iambumblehead
Copy link

No such state exists, anywhere, in any domain of human activity.

The stability of the english language allows you to write such a message here that can be understood by other people.

@guest271314
Copy link

The English language is a bastard, equivocal language that is not remotely "stable". For example, the term "sheriff" came about, not by Anglo-Saxons, but by Normans who conquered Anglo-Saxons and created the officie of "sheriff" as laison between the conqueror and the conquered. Anybody can remix English. It's done every day. From E-40 to Madison Avenue, governments to GitHub with it's "AI-powered" made up slogan, knowing good and well humans make the decisions at GitHub.

@Kreijstal
Copy link

The English language is a bastard, equivocal language that is not remotely "stable". For example, the term "sheriff" came about, not by Anglo-Saxons, but by Normans who conquered Anglo-Saxons and created the officie of "sheriff" as laison between the conqueror and the conquered. Anybody can remix English. It's done every day. From E-40 to Madison Avenue, governments to GitHub with it's "AI-powered" made up slogan, knowing good and well humans make the decisions at GitHub.

it's stable in short timespans, not over millenia, language doesnt change that much in a lifespan.. aka 100 years.

@guest271314
Copy link

Depends. The blues, jazz, rock and roll, hip-hop which is an entire art form in an unto itself all came about in the 20th c., C.E. And all were pointed to as something bad by certain people, until they figured out a way to sell stuff with it. A few weeks ago people were complaining that they couldn't understand anything Kendrick Lamar was saying at the Superbowl halftime show. Good. It ain't for them. Them folks just can't fathom words, culture, art, expression not being developed for them, in their image. And if you examine that deeper you'll find Young and Champollion claiming to have "deciphered" the MTW NTR. That writing and speaking was not made for or by them, and no high priest in the Temple system ever told either of those folks their guesses were correct. Ain't for them. They ain't initiates. Never will be. The best they can do is grave rob and pretend like they know. They never will.

There was a time when the some of the Several States criminalized literacy for Africans held as prisoners-of-war in the fledgling U.S. And that leads into western academia calling African and Turtle Islanders "slaves", which they never were - they were captured in warfare waged against them by European powers. The same as U.S. soldiers captured by Japanese were never called "slaves".

So, I ain't gonna be subject to the whims of somebody trying to control me with their words, when they have disdain for the words my ancestors created. To hell with them and their culture, their thinkling, and their holiest of holies. And I'll still exploit their gear for my own purposes, without a second thought.

So, originally there was just Rhino, a couple others. Then node came about, and a whole bund a people fell in love and they became static and stable - in their own minds. Meanwhile there's dozens of JavaScript runtimes now. There's ECMAScript, ECMA-262, and ECMAScript Modules. It's almost cult-like to stay stuck in 2015 - just because. Whatever. I ain't under that myopic spell, and definitely ain't under the spell of some European colonists with their tricknowledge language that they turn on a dime - always against the natives.

@csvan
Copy link

csvan commented Apr 8, 2025

Oh God this thread <3

@guest271314
Copy link

@csvan

Oh God this thread <3

Perfect example. Are we supposed to know what you mean by "God"?

From https://languages.oup.com/google-dictionary-en

Old English God, of Germanic origin; related to Dutch god and German Gott .

That's a eurocentric term that has its roots in Germany, across the channel.

If the English language was really

stable in short timespans

there wouldn't be in lawsuits over the interpretation of words in laws, contracts. There have been court cases that lasted years turning on the use of "or" in a provision.

@mk-pmb
Copy link

mk-pmb commented Apr 9, 2025

As a side note, what is that "MTW NTR" you were speaking of?

@csvan
Copy link

csvan commented Apr 9, 2025

@guest271314 dunno man, the inherent instability of the English language makes it impossible to understand you, I guess. From what I can gather, you were simply admitting to trolling. You could have been ranting about gods and contracts as well, but it's anyone’s guess at this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment