Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Pure ESM package

Pure ESM package

The package linked to from here is now pure ESM. It cannot be require()'d from CommonJS.

This means you have the following choices:

  1. Use ESM yourself. (preferred)
    Use import foo from 'foo' instead of const foo = require('foo') to import the package. You also need to put "type": "module" in your package.json and more. Follow the below guide.
  2. If the package is used in an async context, you could use await import(…) from CommonJS instead of require(…).
  3. Stay on the existing version of the package until you can move to ESM.

You also need to make sure you're on the latest minor version of Node.js. At minimum Node.js 12.20, 14.14, or 16.0.

I would strongly recommend moving to ESM. ESM can still import CommonJS packages, but CommonJS packages cannot import ESM packages synchronously.

ESM is natively supported by Node.js 12 and later.

You can read more about my ESM plans.

My repos are not the place to ask ESM/TypeScript/Webpack/Jest/ts-node/CRA support questions.

FAQ

How can I move my CommonJS project to ESM?

  • Add "type": "module" to your package.json.
  • Replace "main": "index.js" with "exports": "./index.js" in your package.json.
  • Update the "engines" field in package.json to Node.js 12: "node": "^12.20.0 || ^14.13.1 || >=16.0.0".
  • Remove 'use strict'; from all JavaScript files.
  • Replace all require()/module.export with import/export.
  • Use only full relative file paths for imports: import x from '.';import x from './index.js';.
  • If you have a TypeScript type definition (for example, index.d.ts), update it to use ESM imports/exports.
  • Optional but recommended, use the node: protocol for imports.

Sidenote: If you're looking for guidance on how to add types to your JavaScript package, check out my guide.

Can I import ESM packages in my TypeScript project?

Yes, but you need to convert your project to output ESM. See below.

How can I make my TypeScript project output ESM?

  • Add "type": "module" to your package.json.
  • Replace "main": "index.js" with "exports": "./index.js" in your package.json.
  • Update the "engines" field in package.json to Node.js 12: "node": "^12.20.0 || ^14.13.1 || >=16.0.0".
  • Add "module": "ES2020" to your tsconfig.json.
  • Use only full relative file paths for imports: import x from '.';import x from './index.js';.
  • Remove namespace usage and use export instead.
  • Optional but recommended, use the node: protocol for imports.
  • You must use a .js extension in relative imports even though you're importing .ts files.

If you use ts-node, follow this guide.

How can I import ESM in Electron?

Electron doesn't yet support ESM natively.

You have the following options:

  1. Stay on the previous version of the package in question.
  2. Bundle your dependencies with Webpack into a CommonJS bundle.
  3. Use the esm package.

I'm having problems with ESM and Webpack

The problem is either Webpack or your Webpack configuration. First, ensure you are on the latest version of Webpack. Please don't open an issue on my repo. Try asking on Stack Overflow or open an issue the Webpack repo.

I'm having problems with ESM and Next.js

You must enable the experimental support for ESM.

I'm having problems with ESM and Jest

Read this first. The problem is either Jest (#9771) or your Jest configuration. First, ensure you are on the latest version of Jest. Please don't open an issue on my repo. Try asking on Stack Overflow or open an issue the Jest repo.

I'm having problems with ESM and TypeScript

If you have decided to make your project ESM ("type": "module" in your package.json), make sure you have "module": "ES2020" in your tsconfig.json and that all your import statements to local files use the .js extension, not .ts or no extension.

I'm having problems with ESM and ts-node

Follow this guide and ensure you are on the latest version of ts-node.

I'm having problems with ESM and Create React App

Create React App doesn't yet fully support ESM. I would recommend opening an issue on their repo with the problem you have encountered. One known issue is #10933.

How can I use TypeScript with AVA for an ESM project?

Follow this guide.

How can I make sure I don't accidentally use CommonJS-specific conventions?

We got you covered with this ESLint rule. You should also use this rule.

What do I use instead of __dirname and __filename?

import {fileURLToPath} from 'node:url';
import path from 'node:path';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(fileURLToPath(import.meta.url));

However, in most cases, this is better:

import {fileURLToPath} from 'node:url';

const foo = fileURLToPath(new URL('foo.js', import.meta.url));

And many Node.js APIs accept URL directly, so you can just do this:

const foo = new URL('foo.js', import.meta.url);

How can I import a module and bypass the cache for testing?

There's no good way to do this yet. Not until we get ESM loader hooks. For now, this snippet can be useful:

const importFresh = async modulePath => import(`${modulePath}?x=${new Date()}`);

const chalk = (await importFresh('chalk')).default;

Note: This will cause memory leaks, so only use it for testing, not in production. Also, it will only reload the imported module, not its dependencies.

How can I import JSON?

JavaScript Modules will eventually get native support for JSON, but for now, you can do this:

import {promises as fs} from 'node:fs';

const packageJson = JSON.parse(await fs.readFile('package.json'));

If you target Node.js 14 or later, you can import it using import fs from 'node:fs/promises'; instead.

When should I use a default export or named exports?

My general rule is that if something exports a single main thing, it should be a default export.

Keep in mind that you can combine a default export with named exports when it makes sense:

import readJson, {JSONError} from 'read-json';

Here, we had exported the main thing readJson, but we also exported an error as a named export.

Asynchronous and synchronous API

If your package has both an asynchronous and synchronous main API, I would recommend using named exports:

import {readJson, readJsonSync} from 'read-json';

This makes it clear to the reader that the package exports multiple main APIs. We also follow the Node.js convention of suffixing the synchronous API with Sync.

Readable named exports

I have noticed a bad pattern of packages using overly generic names for named exports:

import {parse} from 'parse-json';

This forces the consumer to either accept the ambiguous name (which might cause naming conflicts) or rename it:

import {parse as parseJson} from 'parse-json';

Instead, make it easy for the user:

import {parseJson} from 'parse-json';

Examples

With ESM, I now prefer descriptive named exports more often than a namespace default export:

CommonJS (before):

const isStream = require('is-stream');

isStream.writable();

ESM (now):

import {isWritableStream} from 'is-stream';

isWritableStream();
@fisker

This comment has been minimized.

Copy link

@fisker fisker commented Apr 16, 2021

Hate and love you, man.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Apr 16, 2021

Been waiting years for this move.

@dangowans

This comment has been minimized.

Copy link

@dangowans dangowans commented Apr 19, 2021

Using this as a "how to" guide to update some of my own packages. Thank you!

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Apr 19, 2021

@sindresorhus

Might want to:

- My repos are not the place to ask ESM/TypeScript/Webpack/Jest/ts-node support question.
+ My repos are not the place to ask ESM/TypeScript/Webpack/Jest/ts-node/CRA support questions.
@sindresorhus

This comment has been minimized.

Copy link
Owner Author

@sindresorhus sindresorhus commented Apr 19, 2021

@qix Done

@cduff

This comment has been minimized.

Copy link

@cduff cduff commented Apr 21, 2021

Hi @sindresorhus,

Thought I'd comment on this with my experience so far.

We're using crypto-random-string, which moved to pure ESM in v4, in a web server project built with typescript and making use of ts-node and ts-node-dev. As per your suggestion, I tried updating my project to ESM but this does not seem possible at the moment:

I don't like the idea of having to move some modules to async dynamic imports but gave it a try. However it doesn't work when TypeScript is configured with "module": "CommonJS" as it translates import() function calls to require().

Staying on old package versions is possible but I worry a time will soon come where it means I can no longer upgrade to realise package improvements/bug fixes.

I'm therefore feeling stuck and not sure of the best path forward on this. So far for us it relates to only one minor dependency but I worry the problem will soon grow much larger.

Any suggestions?

I think I understand your motivation but also wonder about the impact at a time when tooling will struggle to keep up, especially given Node's ESM loader hooks are experimental and changing.

@ujwal-setlur

This comment has been minimized.

Copy link

@ujwal-setlur ujwal-setlur commented Apr 23, 2021

@cduff I agree. Have very similar issues. I love the idea of moving to ESM, but the tools are just not there yet, and I can’t wholesale switch my entire typescript project to ESM quite yet. Unfortunately, for now, I had to switch to d3-random for the random utils. Hopefully the tooling will catch up.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Apr 25, 2021

Usually, moves like this are the kick in the pants to get tooling upgraded to begin with. If neither end does anything, innovation stagnates.

This is a good thing. The pain is growth.

@PopGoesTheWza

This comment has been minimized.

Copy link

@PopGoesTheWza PopGoesTheWza commented Apr 25, 2021

Note that in addition to import statements, you need to also update local export ... from with full relative file paths.

And rather than replacing "main" with "exports" in your package.json, it can be safer to have both. First, "exports" has precedence over "main" anyway; but some tooling may sometime not be fully up-to-date and get confused when there is no "main". I ran into issues with some lerna monorepos when only "export" was present.

@moltar

This comment has been minimized.

Copy link

@moltar moltar commented Apr 27, 2021

Also cannot upgrade multiple projects. Stuck in many directions. There is no path forward 🤷‍♂️

@foray1010

This comment has been minimized.

Copy link

@foray1010 foray1010 commented Apr 28, 2021

Hi @sindresorhus, I suggest:

- Update the "engines" field in package.json to Node.js 12: "node": ">=12".
+ Update the "engines" field in package.json to Node.js 12: "node": ">=12.7".

because:

  1. the exports field is not supported until 12.7.0 (see the diff between 12.6.0 doc and 12.7.0 doc), user with 12.6.0 will be able to install but cannot import this package
  2. You might want >=12.13.0 since it is the first LTS version
  3. You might even want >=12.17.0 since no --experimental-modules flag is required in >=12.17.0 (see the diff between 12.6.0 doc and 12.7.0 doc)
@2c2c

This comment has been minimized.

Copy link

@2c2c 2c2c commented Apr 30, 2021

does there exist a codemod to change any relative imports to add .js file extension? getting burned by dependency update

@oncet

This comment has been minimized.

Copy link

@oncet oncet commented Apr 30, 2021

When integrating with Next.js I had to use next-transpile-module to avoid the Error [ERR_REQUIRE_ESM]: Must use import to load ES Module #28 error (sindresorhus/ky-universal#28).

// next.config.js
const withTM = require("next-transpile-modules")(["ky"]);

module.exports = withTM();
@mesoic

This comment has been minimized.

Copy link

@mesoic mesoic commented May 1, 2021

Add "type": "module" to your package.json.
This can break things

@RDIL

This comment has been minimized.

Copy link

@RDIL RDIL commented May 1, 2021

@mesoic the whole idea of moving from CommonJS to ESM can (and most likely will) break things!

@xavierfoucrier

This comment has been minimized.

Copy link

@xavierfoucrier xavierfoucrier commented May 1, 2021

ESM is the future. Go for it 💗

@akx

This comment has been minimized.

Copy link

@akx akx commented May 4, 2021

Jest can't currently resolve "main"less ESM packages (as these instructions prescribe), so the instructions under "I'm having problems with ESM and Jest" don't necessarily help in the least.

See facebook/jest#11373 & facebook/jest#9771 ...

@sindresorhus

This comment has been minimized.

Copy link
Owner Author

@sindresorhus sindresorhus commented May 5, 2021

@foray1010 Ok. Done.

@mkubilayk

This comment has been minimized.

Copy link

@mkubilayk mkubilayk commented May 6, 2021

Worth noting that node: imports were added in v12.20.0 so "node": ">=12.17" engine restriction doesn't cover that.

@sindresorhus

This comment has been minimized.

Copy link
Owner Author

@sindresorhus sindresorhus commented May 7, 2021

@mkubilayk Good point 👍

@coderaiser

This comment has been minimized.

Copy link

@coderaiser coderaiser commented May 8, 2021

does there exist a codemod to change any relative imports to add .js file extension? getting burned by dependency update

You can use putout code transformer with plugin @putout/plugin-convert-commonjs-to-esm for this purpose, it will convert:

-const {fn} = require('./local');
-module.exports = () => {};
+import {fn} from './local.js';
+export default () => {};
@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 9, 2021

import pkg from '../package.json';

This line results in the following error:

(node:7863) ExperimentalWarning: The ESM module loader is experimental.
internal/modules/run_main.js:54
    internalBinding('errors').triggerUncaughtException(
                              ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".json" for /home/user/files/project/package.json imported from /home/user/files/project/version.js
    at Loader.resolve [as _resolve] (internal/modules/esm/default_resolve.js:126:13)
    at Loader.resolve (internal/modules/esm/loader.js:72:33)
    at Loader.getModuleJob (internal/modules/esm/loader.js:156:40)
    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:42:40)
    at link (internal/modules/esm/module_job.js:41:36) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

@sindresorhus How to import .json files with ESM?

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 9, 2021

@madnight you don't because JSON is not javascript. The hacks for custom file extensions in node have been removed in ESM.

Read it in like a normal file and then parse the JSON. That's what the old CommonJS stuff did anyway.

@coderaiser

This comment has been minimized.

Copy link

@coderaiser coderaiser commented May 9, 2021

@madnight you can use simport for this purpose:

import {createCommons} from 'simport';
const {require} = createCommons(import.meta.url);

const pkg = require('../package.json');
@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 9, 2021

Thx @coderaiser looks good to me, I'll try that.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 9, 2021

@madnight It's really not that difficult, and that's a hack that requires yet another dependency. Just do the right thing? Not that hard.

import {promises as fsp} from 'fs';
const pkg = JSON.parse(await fsp.readFile('package.json', 'utf-8')).

By using a require-type means of importing, you lose out on all of the language benefits of import statements. We're moving away from CommonJS, why are we re-inventing it?

Don't add cruft to the ecosystem please.

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 9, 2021

@Qix-

We're moving away

Moving away would be okay for me, but I don't like to be forced immediately. I have a dependency that can only be included via ESM and I still want to use require at least for some time. I call this transition period. Hard breaking changes are not a good thing (python 2 -> 3), but slow transition is absolutely okay.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 9, 2021

We've been in a transition period for a long time. That's the problem. At some point, some switches need to be flipped.

Remember when we did so with getting people to update from Node 0.10/0.12. We can't perpetually support old systems.

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 9, 2021

If that's true, then it would have been possible for quite some time to use both require and import. Without a hard switch from one to the other.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 9, 2021

Node 12 introduced the imports via a command line flag and was released 2019-04-23. Is two years long enough?

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 9, 2021

So I can simply use Node 12 and require and import works in the same file? Similar like writing python 2 and 3 compatible code in the past?

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 9, 2021

No, because .mjs is different from .js. You're not going to find Python 2/3-like compatibility.

@coderaiser

This comment has been minimized.

Copy link

@coderaiser coderaiser commented May 9, 2021

So I can simply use Node 12 and require and import works in the same file? Similar like writing python 2 and 3 compatible code in the past?

Yes, you can, with help of simport :).
And later you can switch to import assertions when it would be implemented in node(it is available behind the flag), it is stage 3 right now.

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 9, 2021

Then I consider it as breaking change, that breaks your code, without a transition period. I understand that many wait for ESM to happen for a long time. I just want to criticize the way it is introduced now (hard switch).

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 9, 2021

@coderaiser

This comment has been minimized.

Copy link

@coderaiser coderaiser commented May 9, 2021

@coderaiser yes https://github.com/tc39/proposal-import-assertions would be great

Yep, and codemode for transition using putout would be not harder then:

module.exports.report = () => 'Import assertions should be used instead of require';

module.exports.replace = () => ({
    'const __a = require("./package.json")': 'import __a from "./package.json" assert { type: "json" }',
});

Which will transit all codebase at once and clean up unused imports with @putout/plugin-remove-unused-variables as well.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 9, 2021

Then I consider it as breaking change, that breaks your code, without a transition period. I understand that many wait for ESM to happen for a long time. I just want to criticize the way it is introduced now (hard switch).

Please point to where this was a "hard switch". Everything I've seen has been done via major releases, which indicate a break in the public API.

Unless this isn't the case somewhere (it's the case with all of @sindresorhus's modules AFAIK) then this is entirely your problem, not anyone else's. Apologies for the bluntness.

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented May 10, 2021

It's completely okay for me if it is just "my problem". I just haven't expected it that way. I will use the workaround from @coderaiser for a while and then slowly migrate to ESM only and call it a day.

@moltar

This comment has been minimized.

Copy link

@moltar moltar commented May 10, 2021

Read it in like a normal file and then parse the JSON. That's what the old CommonJS stuff did anyway.

Doesn't work that well with TS, as TS was inferring the types from the JSON file automatically. When loading and parsing manually everything is just any.

@clouedoc

This comment has been minimized.

Copy link

@clouedoc clouedoc commented May 10, 2021

Read it in like a normal file and then parse the JSON. That's what the old CommonJS stuff did anyway.

Doesn't work that well with TS, as TS was inferring the types from the JSON file automatically. When loading and parsing manually everything is just any.

Zod is the right tool for this kind of problem.

@moltar

This comment has been minimized.

Copy link

@moltar moltar commented May 10, 2021

Read it in like a normal file and then parse the JSON. That's what the old CommonJS stuff did anyway.

Doesn't work that well with TS, as TS was inferring the types from the JSON file automatically. When loading and parsing manually everything is just any.

Zod is the right tool for this kind of problem.

Really isn't though. Zod is no different than writing types yourself. While TypeScript infers types automatically from imported JSON.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 10, 2021

That's a Typescript problem, not a JavaScript problem.

@clouedoc

This comment has been minimized.

Copy link

@clouedoc clouedoc commented May 10, 2021

Really isn't though...

Let me rephrase it.

Zod is the right tool for this kind of problem because:

  1. You don't get an any blob
  2. You're 100% sure that what the typing system allows you to do will work at runtime - just like type inference from JSON.

If you want to skip the schema writing part, you can combine Quicktype and the extension that someone wrote to convert from interface to Zod schema.

Zod is no different than writing types yourself
It's different because you get to validate the data instead of casting it.

There is no reason to not use Zod when dealing with JSON file, even for file that don't change. You get solid typings and strong validation. The schema-writing overhead is negligible against the time it'll take you to debug a any.foo is undefined, be it when developing or running on a server.

@moltar

This comment has been minimized.

Copy link

@moltar moltar commented May 11, 2021

Zod is the right tool for this kind of problem because:

It is one of the tools. I maintain a large list with benchmarks of this class of tools. Zod actually happens to be one of the slowest, if not the slowest of all of them.

But the tool is for the problem, which I did not have before.

Before, I could just do import myJson from './file.json' and have all of the types automatically inferred.

@shirshak55

This comment has been minimized.

Copy link

@shirshak55 shirshak55 commented May 11, 2021

this seems to be major ecosystem churn unfortunately. Some module doesn't work in esm some works on common js :(

@terribleplan

This comment has been minimized.

Copy link

@terribleplan terribleplan commented May 14, 2021

👎

Node itself isn't mature enough in its implementation, this shouldn't happen until it is. Lack of such maturity means I can't use yarn with PNP, which is important to me. This is the sort of stuff that encourages people to stick to old versions of packages long after they are unmaintained and insecure.

Also referring to "release notes" was confusing to me as they are not very discoverable from npmjs.org (or even github really, it took me a while to understand you even meant to read the "releases" page, I kept looking for a RELEASE_NOTES.md in-repo or something). Maybe a link to this gist at the top of the impacted modules' README would help confused people who's stuff broke from making issues.

Notwithstanding the issue of timing, this is a good goal and the right direction for the ecosystem to move (though I selfishly wish you had gone with migration option #2 from your blog)

@Livven

This comment has been minimized.

Copy link

@Livven Livven commented May 27, 2021

I do want to say that I have tried and failed to migrate several projects using Jest/ts-jest/ts-node to ESM. The linked instructions were generally not very helpful, or required IMO quite invasive changes (e.g. experimental command line flags, unintuitive behavior with TypeScript) that I thought were not worth the effort. All in all, I don't think the ecosystem is ready at this point.

However I can absolutely understand the desire to push the ecosystem forward, since ESM has been a thing for so long now and yet there has been little progress in getting people to adopt it beyond bundling/transpilation 🚀

@moltar Big fan of your benchmarks, even though I still ended up using zod because it has the best combination of API usability and star count IMO 😆 TypeScript's JSON type inference is really quite nice also, and completely removes the need for a runtime checker, might be difficult for people who have not used it to comprehend I guess.

@rhufsky

This comment has been minimized.

Copy link

@rhufsky rhufsky commented May 28, 2021

I really appreciate the move to the more modern ESM, but:

  • I build a Next.js app that uses a lot of packages via import / ESM. The first package I came across that did not work was @sindresorhus/transliterate, telling me that require("@sindresorhus/transliterate") is not supported, so I was a bit confused.
  • Following sindresorhus' advice I googled around for some time and found a solution that worked for me:

npm install next-transpile-modules

next.config.js:

const withTM = require("next-transpile-modules")([
  "escape-string-regexp",
  "@sindresorhus/transliterate",
]); // pass the modules you would like to see transpiled

module.exports = withTM({
  future: {
    webpack5: true, // if you want to use webpack 5
  },
});

For more info, see https://github.com/martpie/next-transpile-modules.

If this solution is acceptable it would be nice if it would show up in the FAQ or somewhere, where us mere mortal Next.js devs would find it easily.

My repos are not the place to ask ESM/TypeScript/Webpack/Jest/ts-node/CRA support questions.

I totally agree, but I also feel that dropping CJS to motivate everybody to move to ESM is a pretty bold move. At least it would be nice to have a few workaround descriptions for the most common environments if something breaks.

@williamdes

This comment has been minimized.

Copy link

@williamdes williamdes commented May 30, 2021

Seems like it breaks mocha ..

Error: Not supported
    at formattedImport (/mnt/Dev/@sudo/gh-deployer/node_modules/mocha/lib/esm-utils.js:7:31)
    at Object.exports.requireOrImport (/mnt/Dev/@sudo/gh-deployer/node_modules/mocha/lib/esm-utils.js:45:14)
    at Object.exports.loadFilesAsync (/mnt/Dev/@sudo/gh-deployer/node_modules/mocha/lib/esm-utils.js:55:34)
    at singleRun (/mnt/Dev/@sudo/gh-deployer/node_modules/mocha/lib/cli/run-helpers.js:125:3)
    at Object.exports.handler (/mnt/Dev/@sudo/gh-deployer/node_modules/mocha/lib/cli/run.js:362:5)

😢
I had to move to https://github.com/klughammer/node-randomstring because I can not afford to spend more time to debug this..
If you have a fix I will gladly go back to your module.
PS: I applied the instructions and the error is the same before and after

Ref: sudo-bot/gh-deployer@f66a1fd#diff-6852641371b653b3743fb3c6a2c46b16f6c8145f3b15c5edebd111b3b1c23ac0

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented May 31, 2021

@williamdes open a ticket with Mocha. That is a Mocha error (look at the paths).

@jpike88

This comment has been minimized.

Copy link

@jpike88 jpike88 commented Jun 4, 2021

Elastic Beanstalk only allows node version of 12.22.1 to be spun up without doing weird backflips, I think this is ESM thing was premature. If anything separate ESM packages should have been released until various environments had time to catch up

@hiendv

This comment has been minimized.

Copy link

@hiendv hiendv commented Jun 4, 2021

Thanks for the note. Waiting for the ecosystem to catch up.

@jpike88

This comment has been minimized.

Copy link

@jpike88 jpike88 commented Jun 4, 2021

Using the simport package worked but meant that the library had to be asynchronously loaded, which meant an ugly async function had to be constructed. Otherwise appears to work fine

const simport = createSimport(__filename);
const PQueue = (await simport('../node_modules/p-queue/dist/index.js'))
		.default;
@sindresorhus

This comment has been minimized.

Copy link
Owner Author

@sindresorhus sindresorhus commented Jun 4, 2021

Elastic Beanstalk only allows node version of 12.22.1 to be spun up without doing weird backflips

Node.js 12.22 fully supports ESM.

@jpike88

This comment has been minimized.

Copy link

@jpike88 jpike88 commented Jun 4, 2021

Yes you're correct sorry, have just been stabbing in the dark

@synox

This comment has been minimized.

Copy link

@synox synox commented Jun 6, 2021

Thanks a lot for this guide! 👏

@williamdes

This comment has been minimized.

Copy link

@williamdes williamdes commented Jun 12, 2021

About the error Error: Not supported, it was already reported at Mocha as mochajs/mocha#4652
I added more comments but still have it with Node 12 and Mocha 9, will edit or post to keep the info up to date for people having the same problem.

@fgarcia

This comment has been minimized.

Copy link

@fgarcia fgarcia commented Jun 25, 2021

According Babel Register

Note: @babel/register does not support compiling native Node.js ES modules on the fly, since currently there is no stable API for intercepting ES modules loading.

So does this mean that any library with "type": "module" is automatically incompatible with projects depending on babel/register??

@promaty

This comment has been minimized.

Copy link

@promaty promaty commented Jun 25, 2021

I just cannot get p-map to work with ts-node, can somebody post a working config 🙏

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Jun 25, 2021

@fgarcia yes, and it's been discouraged from being used like that to begin with for several years now due to the impending modules adoption. Plus, it was never a good thing for performance either, another reason why (at least at one point in time) babel discouraged you from using babel/register for anything other than debugging/development.

@coderaiser

This comment has been minimized.

Copy link

@coderaiser coderaiser commented Jun 26, 2021

So does this mean that any library with "type": "module" is automatically incompatible with projects depending on babel/register??

Yes it is, anyways you can use putout as loader to intercept import in a similar to @babel/register way:

node --loader putout your-file.js

You can also transform input files using Babel Plugins. For example if you need to transform jsx with @babel/plugin-transform-react-jsx you can use.putout.json:

{
    "plugins": [
        "babel/transform-react-jsx"
    ]
}
@lnmunhoz

This comment has been minimized.

Copy link

@lnmunhoz lnmunhoz commented Jun 26, 2021

This is gold. Thank you!

@matey-jack

This comment has been minimized.

Copy link

@matey-jack matey-jack commented Jun 29, 2021

It would be super helpful to have a site like https://python3wos.appspot.com/ to track the ESM compatibility status of the most important (say 10'000) packages on NPM. Then anyone who wants to contribute can create MRs for some of those projects.

Or is there something like that already?

Also, given that we're looking at >50k packages to convert, maybe a little bit of automation could be provided as an easy to use script

npx convert-package-to-esm

which does all the mechanical changes and then prints instructions for the remaining changes that need to be done.

@atao60

This comment has been minimized.

Copy link

@atao60 atao60 commented Jul 1, 2021

I just successfully migrated @atao60/fse-cli to ESM with the help of this gist (the TS part).
Thank you!

@fisker

This comment has been minimized.

Copy link

@fisker fisker commented Jul 1, 2021

Suggestion for the last section:

- import {promises as fs} from 'fs';
+ import {promises as fs} from 'node:fs';
// OR
+ import fs from 'node:fs/promises';

- const packageJson = JSON.parse(await fs.readFile('package.json', 'utf8')).
+ const packageJson = JSON.parse(await fs.readFile('package.json'));

Ref: sindresorhus/eslint-plugin-unicorn#1273

@parzhitsky

This comment has been minimized.

Copy link

@parzhitsky parzhitsky commented Jul 1, 2021

Leap-year start count reached 🙌

@degrootestad

This comment has been minimized.

Copy link

@degrootestad degrootestad commented Jul 9, 2021

Hate and love you, man.

Right! Someone's got to be first. Such a shame we don't move in hordes (or do we? And do they slow us down..?)
Especially the ts-node link looks a bit cumbersome. You're a hero!

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 10, 2021

I appreciate this movement but the ecosystem is not prepared for that yet. All the people who use Typescript for Server Side development have issues with consuming your modules. It's unrealistic to expect that a project can easily switch to ESM. It might be easy from the Node.js perspective but this is only the half story. Currently, I maintain a very small module and it's awful to make it work with your latest p-timeout module. This implicit that all my lib consumers have to enable ESM as well and add a transpile step because in TS you can't import native ESM modules (Gave up after 20min of workarounds see ERR_UNSUPPORTED_DIR_IMPORT).

The right migration would be to support "Dual Package" strategy https://redfin.engineering/node-modules-at-war-why-commonjs-and-es-modules-cant-get-along-9617135eeca1

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Jul 11, 2021

All the people who use Typescript

Take it up with Microsoft. The community has very little influence over Typescript, and it's Typescript's job to be compatible with Javascript, not the other way around.

It might be easy from the Node.js perspective but this is only the half story

I disagree; it's the full story. That's why it's Node.js and not Node.ts or just Node. Node.js is a Javascript engine (V8) that wraps POSIX calls with a few extra bells and whistles. Typescript is a Microsoft project that claims to be compatible with Javascript but of course isn't really.

Take it up with Microsoft if you have problems with Typescript.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 11, 2021

That's why it's Node.js and not Node.ts or just Node ...

This is not how a community works. Typescript is an integral part of the javascript ecosystem. To ignore that means to ignore the now. As mentioned in ESM Plans There is a way to move forward with ESM in a backward-compatible way. From the maintainer perspective of so many packages, it's reasonable to minimize work but the needs should be directed to the consumers. It will end up in a mix, some people will upgrade, some will never and some will provide a CJS fork see. This is way more ugly because the package is no longer maintained by the original author.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Jul 11, 2021

Typescript is an integral part of the javascript ecosystem.

No, it's a completely separate language that Microsoft is shoehorning into the OSS community.

If you find it useful, great. But it is not Javascript. "JavaScript with types" was Flow. Typescript should not - and does not - dictate what happens in Node.js, TC-39, nor any other part of the Javascript ecosystem.

It is Microsoft's job ("job" used here literally, as they are a for-profit entity) to make sure their product is compatible with Javascript if they claim/want it to be so.

it's reasonable to minimize work but the needs should be directed to the consumers

And the headache of creating more and more build steps and hacked together infrastructure to support older systems has a direct, negative impact on consumers. So we agree here.

I've not had any issues, nor have hundreds of other developers I've interacted with. Yes, the tooling isn't great, but it's the web - when has it ever?

Getting increasingly tired with the entitled attitudes telling maintainers what we are and are not allowed to do when we have literal billions of downloads per annum to think about, thousands of dependents, and about a hundred different, conflicting philosophies across a spectrum of skillsets that all seem to crucify us at the first sign of doing things our own way.

Following standards is not a crime. If you're facing issues, that's on you at this point.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 11, 2021

It's not my goal to debate about Microsoft and Typescript. My reason is very simple. I use a lot of packages from sindresorhus because they have good quality. It stopped with that decision because I can't easily switch to ESM. Not less or more. Providing types in the package was a signal to provide first-class typescript support.

@PythonCreator27

This comment has been minimized.

Copy link

@PythonCreator27 PythonCreator27 commented Jul 16, 2021

I appreciate this movement but the ecosystem is not prepared for that yet. All the people who use Typescript for Server Side development have issues with consuming your modules. It's unrealistic to expect that a project can easily switch to ESM. It might be easy from the Node.js perspective but this is only the half story. Currently, I maintain a very small module and it's awful to make it work with your latest p-timeout module. This implicit that all my lib consumers have to enable ESM as well and add a transpile step because in TS you can't import native ESM modules (Gave up after 20min of workarounds see ERR_UNSUPPORTED_DIR_IMPORT).

@StarpTech In native ESM, dir imports are unsupported, so replace abc/something with abc/something/index.js if required. And yes, the .js extension is required. Side note: TS does support native ESM very well.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 16, 2021

TS does support native ESM very well.

@PythonCreator27 try to install and import an ESM module. You can try the latest release of the p-timeout module.

@PythonCreator27

This comment has been minimized.

Copy link

@PythonCreator27 PythonCreator27 commented Jul 16, 2021

@StarpTech You have to have { module: "es2020" } (or higher, like esnext) in your tsconfig. es2015 (or higher) might work, I haven't tested.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 16, 2021

That's the whole point 😄 It doesn't work.

@PythonCreator27

This comment has been minimized.

Copy link

@PythonCreator27 PythonCreator27 commented Jul 16, 2021

@StarpTech It does... it, in a nutshell just requires you to say that you want to use esm, and not cjs as output.

@PythonCreator27

This comment has been minimized.

Copy link

@PythonCreator27 PythonCreator27 commented Jul 16, 2021

Just an update, the latest next.js canary seems to have some support for using pure ESM modules (experimentally of course). See vercel/next.js#27069.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 16, 2021

@PythonCreator27 please provide an example or disconfirm my example. We are talking about native ESM no transpiling / bundler involved.

@PythonCreator27

This comment has been minimized.

@atao60

This comment has been minimized.

Copy link

@atao60 atao60 commented Jul 17, 2021

@StarpTech I'm not quite sure to understand the 'no transpiling part' of native ESM no transpiling / bundler involved. As soon as some code is in TS, how can you avoid some kind of transpilation? I.e. no babel and no TSC?

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 17, 2021

Thank you for the example. Indeed it works. Not sure, why my own example didn't work.

The fact that all upstream consumers need to understand ESM natively is still huge. I'd still prefer a compatibility layer with package exports. Very easy to do with https://github.com/lukeed/bundt.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Jul 17, 2021

The fact that all upstream consumers need to understand ESM natively is still huge.

They explicitly do not. CommonJS is still supported by ESModules. Upstream CommonJS modules can still be used even today.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 17, 2021

You explained it in the reverse direction. CommonJS is supported in ESM but a Node.js user who uses CJS can't use ESM. That's why I'd need to provide a compatibility layer.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 17, 2021

@atao60 that was a bit misleading. I mean the "target" module system is supported natively by the platform.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Jul 17, 2021

"Upstream" means "things I depend on" (loosely). An ESModules project can use CommonJS upstream dependencies just fine. That was my point. I'm not really sure what we're debating about anymore 😅

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Jul 17, 2021

My single point is that going full ESM is a very hard break because CJS is still the major module system in Node.js. Making that decision from an application perspective is more rational than from a library with a lot of different consumers. Especially, when they already use your library and can no longer upgrade.

@mekwall

This comment has been minimized.

Copy link

@mekwall mekwall commented Jul 18, 2021

I believe this move was done too early. @sindresorhus Is there a reason you didn't go the middle way with a hybrid package (providing both esm and cjs modules)?

@quetzalsly

This comment has been minimized.

Copy link

@quetzalsly quetzalsly commented Jul 18, 2021

this is incredibly stupid, yeah let me just spend an entire day changing all of the imports in my 10k lines of code project, or... just dont upgrade ez.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Jul 18, 2021

@quetzalsly Please be respectful in your discourse.

@mekwall

This comment has been minimized.

Copy link

@mekwall mekwall commented Jul 20, 2021

I've created a hybrid package of p-queue over at https://github.com/mekwall/p-queue, mainly because I use it a lot and want to keep up to date with the latest fixes and changes, but also as a proof of concept to share with the rest of you. I hope that @sindresorhus will revisit this and rethink his abandonment of CommonJS.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Jul 20, 2021

@mekwall CommonJS is deprecated. It does nobody any good to keep it alive.

@fgarcia

This comment has been minimized.

Copy link

@fgarcia fgarcia commented Jul 20, 2021

this is the kind of thing in which I am glad we all can move into something better... and then shed a tear for all those people working in projects where they do not have voice / power to tell everyone else to upgrade their code to the new age. I am on the lucky side. At work we were told to stop upgrading esm-only packages until further notice.

On a personal project, I wanted to change my markdown engine, but it breaks my testing / code reload dependency on @babel/register. I am waiting until Babel itself supports esm only libraries... or upgrade my testing/development pipeline to continuously re-compile each file

@mekwall

This comment has been minimized.

Copy link

@mekwall mekwall commented Jul 20, 2021

@mekwall CommonJS is deprecated. It does nobody any good to keep it alive.

I am not talking about keeping it alive. I just want a nice and smooth transition. I am all for ESM but it's obviously not as easy as to dump support for CommonJS in a move like this.

@mekwall

This comment has been minimized.

Copy link

@mekwall mekwall commented Jul 20, 2021

this is the kind of thing in which I am glad we all can move into something better... and then shed a tear for all those people working in projects where they do not have voice / power to tell everyone else to upgrade their code to the new age. I am on the lucky side. At work we were told to stop upgrading esm-only packages until further notice.

On a personal project, I wanted to change my markdown engine, but it breaks my testing / code reload dependency on @babel/register. I am waiting until Babel itself supports esm only libraries... or upgrade my testing/development pipeline to continuously re-compile each file

I have a lot of say in our projects, but I don't control third-party dependencies, and right now support for ESM is not that good. It's getting there and I wish it could go faster, but until we're there I think we should all help out with the transition.

@dlong500

This comment has been minimized.

Copy link

@dlong500 dlong500 commented Jul 21, 2021

I have to say that, while I absolutely support your efforts to push the javascript ecosystem forward, this move to offering only pure ESM and not a "dual" output should have waited until Node.js releases a stable loaders hook API for ESM. It's marked experimental even in the latest Node.js 16.5.0. This limits what build tooling can do with ESM. And yes, there are still valid use cases for bundling/tooling outside of transpilation (though even transpilation will still be needed for future ES features).

Case in point: Babel can't perform on-demand processing of ESM type projects because of the Node.js hook API issue. So neither @babel/register or @babel-node can work for development environments with a package of type module. Developers will have to compile (as if in production) even in development environments until this Node.js loader API issue is resolved.

Maybe the Node.js ESM loader hook API will stabilize quickly, tooling will catch up, and this will become a moot point. But right now it's a bit of a pain (and that makes it more work for people to move their own projects to ESM). That could end up backfiring with a large percentage of users choosing to just lock-in older non-ESM versions of packages.

@msikma

This comment has been minimized.

Copy link

@msikma msikma commented Jul 24, 2021

With respect, but I consider this a poor and inconsiderate move. It's your package, and you can do whatever you want with it, but you're basically saying "I like ESM, therefore you should all use it and my packages are now unavailable to you until you do."

There isn't actually any hard need for this package (string-width) to be pure ESM. It doesn't use TLA. This is just a way to force people to move up their migration schedule regardless of the circumstances. Migrating away from CommonJS can potentially be very laborious, and not necessarily for any real gain. require() will likely never be removed because of how colossal the disruption would be to the ecosystem.

ESM is completely unusable in a CJS script. If you want to use even the tiniest module in a CJS project you need to convert it all, no matter how many thousands of files your project uses. If you're targeting a legacy version of Node (which is a perfectly fine use case), you're totally out of luck.

You might be a in a position where that isn't a problem, but not everybody has that luxury. Library authors start marking their projects as ESM only without a hard need in an attempt to "push the community forward" is our version of the Python 3 migration, except even more disruptive.

@williamdes

This comment has been minimized.

Copy link

@williamdes williamdes commented Jul 24, 2021

I totally agree with @msikma
I had to replace the package with another one because it was impossible to make it work and I had not two days to waste figuring out what was wrong to only generate random strings.
Yeah it feels wrong but this is also the decision I had to make in order to continue moving on given the time I had and that nobody funds my work so I am taking free time to figure out how I can make all my tools work with this ESM only package.

PS: this is not a mean comment, just a story telling

If you want to better understand how it felt to have an ESM only package, look at this gif ;)

https://tenor.com/view/doctor-reflexes-gif-5270270

doctor-reflexes

@rauschma

This comment has been minimized.

Copy link

@rauschma rauschma commented Jul 26, 2021

In order to get TypeScript to work when I use "exports", I needed to also use "typesVersions": https://2ality.com/2021/06/typescript-esm-nodejs.html#package-exports%3A-hiding-package-internals-and-providing-nicer-module-specifiers

@Urigo

This comment has been minimized.

Copy link

@Urigo Urigo commented Jul 26, 2021

Did anyone try upgrading to globby 12 in gatsby-node.js?

@RayhanADev

This comment has been minimized.

Copy link

@RayhanADev RayhanADev commented Jul 29, 2021

<3 Love what you do for the community dude!

@igorskyflyer

This comment has been minimized.

Copy link

@igorskyflyer igorskyflyer commented Jul 31, 2021

Hey @sindresorhus thanks for the guide, a life-saver, mate! I've spent the whole fricking day (yes, I'm pissed 😅) trying to get a CommonJS library (my own of course) to work properly when converted to a ES module. I was thinking of converting all my npm packages to ES modules because as far as I know, only ES packages provide enough information for bundlers to properly do tree shaking which reduces the final bundle size substantially. Anybody have some information about that? Are ES modules an overkill for that because CommonJS modules don't support tree shaking (or poorly support) and I personally do prefer the syntax of ES modules to begin with.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Aug 1, 2021

Or just upgrade...? It would be less work to do that than maintain old, deprecated standards.

@dlong500

This comment has been minimized.

Copy link

@dlong500 dlong500 commented Aug 1, 2021

@qix, I don't think forking all the packages is the way to go, but saying "just upgrade" as though it's a few minutes of work (regardless of how complex or large a project) is exactly the mentality that bothers me. Moving to pure ESM is a noble goal and is certainly the future, but there ARE hurdles and tooling issues right now. Why not provide dual ESM/CJS packages as an intermediate step while these issues get ironed out? That doesn't stop anyone who wants to go full ESM in their own projects from doing so, but it also allows for less headache for everyone else. There's no reason to force anything right now--at least not until Node.js and the major processing tools work out a few remaining kinks. Forcing the transition will just cause frustration and likely lead to many developers locking on old CJS-compatible versions. Once the ecosystem is a bit more mature then it makes sense to drop CJS entirely. I just don't think we're there yet.

@alexp1917

This comment has been minimized.

Copy link

@alexp1917 alexp1917 commented Aug 1, 2021

this guide is missing a critical explanation of this link on which took me ~3 hrs to find: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#dynamic_imports -- this allows CJS modules to require "import" only modules.

@lostpebble

This comment has been minimized.

Copy link

@lostpebble lostpebble commented Aug 5, 2021

@Qix- , I assure you that there are many people out there who do want to move over to pure ESM for their code, but the very tools they use to build their code are still holding them back at the moment- be it because they're for some reason still locked to an old version (plugin incompatibilities) or that the tool itself doesn't support ESM yet. Its definitely not helping saying to people to "just accept it and upgrade"- we do want to already, but realistically- such fundamental changes take time. The JavaScript ecosystem is a huge slow-moving beast which many of us are unfortunately pinned to in a multitude of ways- it doesn't help anyone if you run out ahead and shout back at us to "just run faster, already!"...

@igorskyflyer

This comment has been minimized.

Copy link

@igorskyflyer igorskyflyer commented Aug 5, 2021

@Qix- , I assure you that there are many people out there who do want to move over to pure ESM for their code, but the very tools they use to build their code are still holding them back at the moment- be it because they're for some reason still locked to an old version (plugin incompatibilities) or that the tool itself doesn't support ESM yet. Its definitely not helping saying to people to "just accept it and upgrade"- we do want to already, but realistically- such fundamental changes take time. The JavaScript ecosystem is a huge slow-moving beast which many of us are unfortunately pinned to in a multitude of ways- it doesn't help anyone if you run out ahead and shout back at us to "just run faster, already!"...

A nice discussion about this is available at this TypeScript issue. Anyone interested can check it out and comment their opinion. Together we can make it work faster. 🤗

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Aug 5, 2021

or that the tool itself doesn't support ESM yet

Then don't upgrade. That's why we have semver.

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented Aug 6, 2021

Then don't upgrade.

This is what people were telling you in case you couldn't update to python 3. It led to a complete disaster for more than a decade, where the whole community was split on either python 2.7 or python 3.

@lostpebble

This comment has been minimized.

Copy link

@lostpebble lostpebble commented Aug 6, 2021

Then don't upgrade. That's why we have semver.

You still don't get it. Most issues we will run into down the line won't be because we choose not to upgrade- other projects also want to be at the latest version (security releases and more features), JavaScript is a very deep and interconnected ecosystem, and a lot of deep dependencies will upgrade- and this will break our stacks. I don't get why this is so hard for you to understand- this whole thread you've just deliberately ignored the very real concerns of people.

I don't think you understand the tax that JavaScript has already put on people's lives when it comes to configuration- if you're only working on new projects every time and can afford a clean templated slate- good for you! But it's actually insane how much stress trying to get certain stacks to work creates. Now for the sake of being "ahead of the curve" you're going to create a whole new brand of headaches for folks- which is actually an unsolvable dead-end (there is no way for us to make commonjs work with module-only modules).

What folks here have just been trying to say is: we all want to move forward as well, BUT making something "future or nothing" when the JavaScript ecosystem is still sitting in its present state is very premature and will hurt a lot of developers' time and sanity.

@Qix-

This comment has been minimized.

Copy link

@Qix- Qix- commented Aug 6, 2021

I don't think you understand the tax that JavaScript has already put on people's lives when it comes to configuration

I do. I've been writing Javascript for almost 20 years, and I've been a part of the Node community since 0.10 was released. I fully understand how all of this works.

If you can't upgrade, that's not great. But upgrades aren't free. There is no way, in any language, in any ecosystem, to make breaking changes in a backwards compatible way. It's an oxymoron.

Which is why versioning systems like Semver exist. Major releases are major releases. You need to make sure the breaking changes in a major release are compatible with your codebase, and if they aren't, and you still wish to upgrade, then you need to make changes to your codebase.

I don't see why this is so difficult to understand. By using Node.js, you opted in to the Semver paradigm. That paradigm is laid out cleanly at semver.org for reference. It is what is used ecosystem-wide. We're allowed to make these exact sorts of changes thanks to Semver.

This is what people were telling you in case you couldn't update to python 3.

No, what I'm saying is the direct opposite of the Python3 fiasco. The library maintainers are pushing forward with new versions of the languages. You're free to upgrade on your own time by simply not using new major versions until you're ready.

It is not our job to manage your projects. It's something you need to learn to do yourself. This is how it has been since the dawn of software.

@madnight

This comment has been minimized.

Copy link

@madnight madnight commented Aug 6, 2021

No, what I'm saying is the direct opposite of the Python3 fiasco.

No, I completely disagree with you. Because, you are absolutely argumenting in the same way that python 3 advocates have been argumenting.

You're free to upgrade on your own time by simply not using new major versions until you're ready.

This is exactly the reason why we had 18 patch levels (2.7.18) of python 2.7 over a decade.

@dlong500

This comment has been minimized.

Copy link

@dlong500 dlong500 commented Aug 6, 2021

If you can't upgrade, that's not great. But upgrades aren't free. There is no way, in any language, in any ecosystem, to make breaking changes in a backwards compatible way. It's an oxymoron.

I'm trying to be respectful of various opinions on the matter, but this mentality is not helpful. No one is asking for "breaking changes" to be backwards compatible. All anyone is asking at this point is that ESM isn't forced too soon. Since it is quite easy for libraries to produce a dual ESM/CJS build you can offer newer features and bugfixes to both users who are able to go pure ESM as well as those stuck on CJS for the time being. Semver doesn't help if there are new features or security fixes after the change to pure ESM.

I really don't get the negativity around the dual package strategy. Once build tools catch up it won't be necessary (especially because I'm sure there will be a way to transpile ESM back to CJS in a build pipeline at some point). But as I said before we aren't there yet.

@robertying

This comment has been minimized.

Copy link

@robertying robertying commented Aug 8, 2021

If your packages have lots of dependencies which are ESM and you are using next-transpile-modules, try using a script to find all dependencies that are ESM:

const fs = require("fs");
const path = require("path");

const list = [];

fs.readdirSync("node_modules", { withFileTypes: true })
  .filter((d) => d.isDirectory())
  .forEach((d) => {
    try {
      const info = JSON.parse(
        fs.readFileSync(
          path.join("node_modules", d.name, "package.json"),
          "utf-8"
        )
      );
      if (info.type === "module") {
        list.push(d.name);
      }
    } catch {}
  });

console.log(list);

So that you don't have to test, update module list and restart dev server multiple times to make all modules transpiled.

@wooorm

This comment has been minimized.

Copy link

@wooorm wooorm commented Aug 8, 2021

@dlong500 In my experience as an open source maintainer of billions of downloads: Dual CJS/ESM doesn’t work. I’ve introduced bugs for downstream users with that because existing tools don’t handle dual ESM at all. They handle CJS and faux-ESM, but not spec compliant ESM. There are much less problems for users by picking one or the other. Example: remarkjs/react-markdown#518.

Your whole argument also seems to be: yes we should do it but not now because my code broke. That’s subjective. That’s time. You can wait a bit for your tools to start working and the time being right. But those tools are starting to finally work because maintainers are moving.

Finally, ESM is the future. It’s going to happen. There’s really no point in arguing with open source maintainers you aren’t paying about them moving towards spec compliance.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Aug 8, 2021

Dual CJS/ESM doesn’t work. I’ve introduced bugs for downstream users with that because existing tools don’t handle dual ESM at all

This is also very subjective. I also know many popular projects that use it successfully.

There are also projects (e.g globby) that only address Node.js where no bundler/postprocessing tools are involved.

I think we all can agree that ESM is the future but there is a difference between a migration and a hard cut-off. It's a fact that I can no longer rely on fixes without upgrading my project to ESM. This will eventually happen in weeks or months. This isn't user-friendly it's enforcement at all cost and it is not absolutely necessary. @dlong500 explained it very well.

@dlong500

This comment has been minimized.

Copy link

@dlong500 dlong500 commented Aug 8, 2021

@wooorm You don't seem to be getting my argument at all. I'm absolutely NOT saying we shouldn't do it now. I'm saying we should do it now in a transitional manner (as sindresorhus himself listed as one of the options in his blog in mentioning the dual strategy). Your mention of the tooling problems around dual CJS/ESM is a perfect example of exactly what I'm getting at. There are tooling problems related to ESM (both with dual target packages as well as packages that have gone pure ESM). The tooling is exactly what is causing problems now (not CJS, not ESM).

There is no technical reason why a dual package strategy can't work, nor is there a reason pure ESM packages can't work. It's all about the tooling catching up. Why not allow the tooling to catch up before forcing pure ESM libraries on everyone? Dual package IMO is the best because those who can't yet transition their own projects to ESM will typically have things working as they have been under CJS. With ESM being the newer spec, the dual package strategy still gives those who want to use it now a viable option (even if there are still some tooling kinks to work out).

As for your final statement, has anyone in this thread said ESM isn't the future? Not helpful at all to drone on about things we all know are obvious. And of course we can respectfully argue with open source maintainers. It's valid feedback. They aren't obligated to do anything or even listen to anyone, but we can all have the same goal with different ideas of how to get there. Acting like anyone who disagrees with a decision has no right to even voice an opinion just because they aren't a maintainer or paying for something is just silly. That being said, I think this issue has run it's course. The maintainers will do what they will do, and that's fine. I was simply voicing my opinion that libraries moving to pure ESM too fast will likely cause headaches that actually work against the goals of moving the javascript ecosystem forward.

@jaydenseric

This comment has been minimized.

Copy link

@jaydenseric jaydenseric commented Aug 9, 2021

Most people don't realize that their imagination of a package that perfectly supports both ESM (for Node.js, Deno/browsers, and bundlers) and CJS consumers is impossible.

To avoid the dual package hazard, all the package's modules must be CJS with a single "default export" (module.exports =), with the exception of index files that are conditionally resolved for ESM consumers (via conditional exports) to allow proper named exports. This setup means everyone is running and bundling the same code, regardless if it's imported or required. It's incorrect for a package to publish the same API built twice; once to ESM files, and another time to CJS files. @StarpTech it appears all of the "popular projects that use it successfully" you listed are incorrect. If part of your codebase requires from them, and another imports the same thing, duplicated code will run/bundle.

Here's a correct example:

The downsides are:

  • Few people understand why it should be done this way, or how to do it correctly. The concepts are very abstract and package authors have a habit of publishing something that happens to work when they tested it locally for their particular projects, even if it's horribly broken for other kinds of projects. > 80% of published dual ESM/CJS packages cause the dual package hazard and most of the time the authors (doesn't really matter how famous they are) don't even know what that is.
  • The package author has to write and maintain primarily CJS code. This sucks, because:
    • You need to make sure all your dev tools support ESM and CJS modules, and can tell what the mode should be for any given file. Which they largely fail at even though tool authors have had years to get it right. Linters, etc. still suck at detecting the module type without elaborate manual config. The TypeScript team have purposefully sabotaged .mjs support! Tooling that needs to parse the source code (e.g. via Babel) to work has to have double the complexity in some cases because the AST for a CJS module (require calls, etc.) is very different to an ESM module (import statements, etc.).
    • You can't use any pure ESM dependencies, because your modules are CJS. This downside alone is a dealbreaker.
  • The package install size is slightly larger because index modules have to exist as both ESM and CJS files.
  • Your packages are only ESM on the surface.
    • You can't import any of the code in a browser or Deno, because the modules are all either CJS, or ESM that imports CJS and these runtimes can't deal with CJS at all.
    • CDN's that allow packages to be imported via URL for browsers or Deno have to do more fiddly work to convert all the CJS to serve pure ESM. This results in bugs and increases the barrier to entry for people that want to build or host their own.

IMO the JavaScript community has had the priorities wrong for a long time; we need to prioritize technical elegance, simplicity, DX, and documentation for package authors over package consumers. Otherwise we are building our castles on top of a foundation of mud only pigs enjoy working in.

I remember wishing 5 years ago that Node.js would just deprecate CJS and support ESM from the next major version. If only that had happened when the npm ecosystem was much smaller; literally hundreds, perhaps thousands of hours of my life have been wasted on CJS issues.

There is never going to be a time that it's painless to switch to pure ESM. The faster we rip the bandaid off and focus on publishing simple, standards-aligned packages for us all to build on, the better. All the people complaining have no idea that Deno is going to disrupt their entire world soon anyway, a runtime that only works with real ESM; browser-like HTTP/S imports.

@dlong500

This comment has been minimized.

Copy link

@dlong500 dlong500 commented Aug 9, 2021

@jaydenseric I'll just briefly respond that you have some very valid points. What I'd say in response is that it might be nice if some of the major players who want to push ESM usage forward could help with the tooling in the ecosystem and not just the libraries. As I've said many times before, once the tooling catches up most of the issues with pure ESM go away. There will always be some effort involved in the transition, but a lot of people (myself included) are caught in a virtual paradox of updates that break apps, trying to fix the issues which leads to more broken things, and finally rolling back to an older version to work on a fix at a later time.

@jaydenseric

This comment has been minimized.

Copy link

@jaydenseric jaydenseric commented Aug 9, 2021

@dlong500

it might be nice if some of the major players who want to push ESM usage forward could help with the tooling in the ecosystem and not just the libraries

I used to think that we should wait for tools to be ready before republishing everything as pure ESM. But in reality, some of us have spent years attempting to raise issues and PRs fixing native ESM compat for big mainstream packages, to be met with a total lack of urgency or even outright hostility by project owners. We've been pushing Next.js for over 3 years to support native ESM (both in project code, and for dependencies) and there was zero prioritization until @sindresorhus lead the wave of publishing pure ESM anyway, forcing them to make some progress in canary releases in just the last few weeks. Here is a styled-jsx PR I raised 15 months ago to support Node.js ESM that still hasn't been merged.

Unfortunately the npm ecosystem doesn't prioritize the right kinds of things. Big budget brands build and market huge, monolithic frameworks with no separations of concerns that lock users in. Normies just install things based off of brand recognition, stars, and install counts so it's a vicious cycle that gets worse and worse.

If you're willing to look for lightweight, standards-aligned dev tools that have narrow focus (separation of concerns, so no lock-in) you will be pleasantly surprised that you can do almost anything you need with support for native ESM. For example, Jest (currently a 24.9 MB install size!) for years has been plagued with native ESM issues, along with all the other 10k+ star alternatives. So long ago I took months off work to publish a set of tiny standalone tools that have worked flawlessly to test all my own packages (that get millions of installs, so not toys):

After all these years they've only accumulated 21 stars combined. As they say; "you can lead a horse to water, but you can't make it drink". I even experienced cyber-bullying (since deleted and apologized for) for creating coverage-node, because it threatened the status quo for packages overlapping in purpose.

It's a bit like the Internet Explorer situation. If we support it because lazy orgs still use it, then said orgs won't update from it. Ad infinitum.

@wooorm

This comment has been minimized.

Copy link

@wooorm wooorm commented Aug 9, 2021

@StarpTech @dlong500 Two more issues that show that dual doesn’t work well, along with the one I mentioned before in https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c#gistcomment-3850692 are transloadit/uppy#1513 and Rich-Harris/periscopic#10.

Observe that in these cases the dual package is package A, which is used a certain way in package B, and finally used in application C with a certain bundler. Bundlers “randomly” switch from one format of A to the other (and sometimes adding faux-esm to the mix) based on how application C is configured, breaking how package B uses package A.

You see, if a maintainer publishes an ESM-only package, it affects their direct dependants (in a semver major). If its dual, then it would affect transient dependants.
In two scenarios:

  • a) the maintainer might mistakingly think that changing from CJS to dual could be done in a minor version, as I did with micromark, causing transient users’ codebases to start breaking, or
  • b) it would be done in a major, but a direct user of the dual package would release it in a minor (tests passed nothing broke for them so 🤷‍♂️), still causing transient dependents codebases to start breaking

Plus the dual-package hazard as explained by @jaydenseric.
And adding the complexity of maintaining different release lines.

Combined I’d say that the ecosystem would me much worse off if the type of packages that Sindre and I maintain were dual.

@StarpTech

This comment has been minimized.

Copy link

@StarpTech StarpTech commented Aug 9, 2021

@wooorm I think we should differentiate between server-side only packages and the browser. In the browser world, you have tooling that guarantees a kind of mixed usage in both directions ESM/CJS, on the server you can only rely on the native module system.

@llldar

This comment has been minimized.

Copy link

@llldar llldar commented Aug 10, 2021

why not use typescript as source and with 2 configs

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "module": "commonjs",
    "outDir": "dist"
  }
}

and

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "moduleResolution": "node",
    "module": "es2015",
    "outDir": "dist-esm"
  }
}

you instantly get support for both commonjs and esm.

@followbl

This comment has been minimized.

Copy link

@followbl followbl commented Aug 11, 2021

@sindresorhus next works with your esm modules in "next": "11.1.0" with the following added to your next.config

  experimental: {
    esmExternals: true,
  },

anyone can find a working example here... https://github.com/followbl/esm-exp

read more here...https://nextjs.org/blog/next-11-1#es-modules-support

@ChristianMurphy

This comment has been minimized.

Copy link

@ChristianMurphy ChristianMurphy commented Aug 15, 2021

Another build tool which can require workarounds, esbuild.
specifically ESBuild bundling ESM for Node (esbuild --format=esm --platform=node --bundle) does not transpile requires into imports, or patch in a require function.

resulting in

throw new Error('Dynamic require of "' + x + '" is not supported');
        ^

Error: Dynamic require of "{module name}" is not supported

errors at runtime.

In most cases these can be addressed by adding a banner which creates a require function based off import.meta.url

esbuild --format=esm --platform=node --bundle --banner:js=\"import {createRequire} from 'module'\nconst require = createRequire(import.meta.url)\"

or

 await esbuild.build({
   bundle: true,
   platform: "node",
   format: "esm",
+  banner: {
+    js: `
+      import {createRequire} from 'module'
+      const require = createRequire(import.meta.url)
+    `,
+  },
 });

credit for solution: evanw/esbuild#1232 (comment) and evanw/esbuild#946 (comment)

@leerob

This comment has been minimized.

Copy link

@leerob leerob commented Aug 16, 2021

Re: Next.js, it's currently linked to this workaround. As of 11.1, there is now experimental support for direct ESM support. Try it out!

https://nextjs.org/11-1

@Urigo

This comment has been minimized.

Copy link

@Urigo Urigo commented Aug 16, 2021

@leerob here is an example of trying to deploy a NextJS 11.1 website with globby 12.
It is currently not possible and I really hope things like that could be solved soon as it is a very common use case

@xobotyi

This comment has been minimized.

Copy link

@xobotyi xobotyi commented Aug 19, 2021

Very bad desicion especially for TS users 😕

@magicdawn

This comment has been minimized.

Copy link

@magicdawn magicdawn commented Aug 20, 2021

You must use a .js extension in relative imports even though you're importing .ts files.

It's super ugly to write import x from './x.js', even there is no x.js, but x.ts

I believe that's what we are talking about "not mature enough"

Agree with this: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c#gistcomment-3742287

@sindresorhus

This comment has been minimized.

Copy link
Owner Author

@sindresorhus sindresorhus commented Aug 20, 2021

It's super ugly to write import x from './x.js', even there is no x.js, but x.ts

This is a limitation of TypeScript, not ESM or Node.js.

@weyert

This comment has been minimized.

Copy link

@weyert weyert commented Aug 23, 2021

Thank you really resourceful document. I am only wondering how well instrumenting ESM modules is supported as the commonly used require-in-the-middle-package won't work with it.

That's my main concern I would like to have tracing to work. Of course, best solution would to somehow convince the popular package maintainers to instrument their packages with Opentelemetry.

@wooorm

This comment has been minimized.

Copy link

@wooorm wooorm commented Aug 23, 2021

Maybe https://github.com/DataDog/import-in-the-middle, which is a few days old, can help there?

@sindresorhus

This comment has been minimized.

Copy link
Owner Author

@sindresorhus sindresorhus commented Aug 24, 2021

That's my main concern I would like to have tracing to work. Of course, best solution would to somehow convince the popular package maintainers to instrument their packages with Opentelemetry.

Actually, the best way would be to convince them to support diagnostics_channel: https://twitter.com/bengl/status/1429908407755149324

@weyert

This comment has been minimized.

Copy link

@weyert weyert commented Aug 24, 2021

I assume it works happily together with async_hooks? Looks like there has been some discussion about it open-telemetry/opentelemetry-js#1263 and graphql/graphql-js#3133

Oh vaguely heard about diagnostics_channel. I should read up more about. I like the idea of using an existing standard for tracing, metrics, logs, and RUM that makes it easy to switch between providers :)

@JaneJeon

This comment has been minimized.

Copy link

@JaneJeon JaneJeon commented Aug 24, 2021

This is freaking PAINFUL. Neither webpack nor babel works to compile down libraries like normalize-url/got from ESM down into CJS, and testing support for ESM is just... ugh

I wasted an entire week trying to get this set up, because apparently none of the tutorials online, none of the examples shown by sindre here works...

@lifeiscontent

This comment has been minimized.

Copy link

@lifeiscontent lifeiscontent commented Aug 26, 2021

@sindresorhus for typescript + eslint do you know how to configure the import/no-unresolved to allow for import of .js files?

@dangowans

This comment has been minimized.

Copy link

@dangowans dangowans commented Aug 27, 2021

@lifeiscontent

@ sindresorhus for typescript + eslint do you know how to configure the import/no-unresolved to allow for import of .js files?

My .eslintrc.json "extends" block includes plugin:import/typescript, excerpt below.

"extends": [
    "eslint:recommended",
    "plugin:import/recommended",
    "plugin:import/typescript",
]
@jpike88

This comment has been minimized.

Copy link

@jpike88 jpike88 commented Aug 28, 2021

For those still having drama, use simport, I'll paste the code that worked for me again:

const simport = createSimport(__filename);
const PQueue = (await simport('../node_modules/p-queue/dist/index.js'))
		.default;
@brunolcarlos

This comment has been minimized.

Copy link

@brunolcarlos brunolcarlos commented Aug 28, 2021

I'm using the last react version and i'm getting the error for import

@rauschma

This comment has been minimized.

Copy link

@rauschma rauschma commented Aug 28, 2021

@brunolcarlos: React gave me much grief in my ESM-only project. I switched to Preact and it worked fine.

This may also be an option: https://github.com/snowpackjs/react