Skip to content

Instantly share code, notes, and snippets.

@hsablonniere
Last active November 15, 2021 21:19
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hsablonniere/ec5e70ca920093010493e556d50dba03 to your computer and use it in GitHub Desktop.
Save hsablonniere/ec5e70ca920093010493e556d50dba03 to your computer and use it in GitHub Desktop.
Explainer for my rollup plugin idea about import.meta.url with assets

Explainer for my rollup plugin idea

The "i-prefer-import-meta-url-over-using-esm-import-to-get-a-relative-file-url" plugin ;-)

Context

In my components library, I have SVG images. I used them with <img> tags in my components and to get the proper relative URL, I use new URL('../assets/image.svg', import.meta.url').href.

To make this work in a rollup build, you need to:

  • Preserve relative file tree structure of your JavaScript files
  • Copy files with rollup-plugin-copy and keep same relative tree structure

I had to rework my tree structure to have all components in src/foo/component-foo.js and all assets in src/assets/image.svg. I also had to make sure my components end up in src/dist/component-foo.js and copied assets in dist/assets/image.svg.

It works "OK" in my situation but it has a few limitations:

  • The rules on the tree structure are too strict
  • When I did a test to emit just one bundle instead of preserving modules, I had to emit it in a subdir so the ../assets still work.
  • If a relative path does not exist, the build does not fail.
  • You need the copy plugin.

Why don't you use ESM import for your assets?

"Everyone is doing it", especially with Webpack loaders but it's not standard. It requires specific bundler config to work before serving the source files to a browser.

With import.meta.url it works as is in the browser before the bundling. This plugin idea is "just a way" to help rollup understand this pattern better and rewrite stuffs if the relative tree structure is changed.

A solution

A tranform plugin (with transform() hook) that:

  • detects new URL('../path/to.svg', import.meta.url)
  • emits file with this.emitFile({type:'asset', /* .... */})
  • replaces the new URL('../path/to.svg', import.meta.url) with import.meta.ROLLUP_FILE_URL_referenceId so rollup can replace it with the correct relative path (and maybe the hash).

Proof of concept

Here's the untested/dirty/hacky proof of concept:

// rollup.config.js

export default {
  // ...
  plugins: [
    // ...
    {
      transform(code, id) {
        const rgx = /new URL\('(.*)', import\.meta\.url\)\.href/g;
        const matches = Array.from(code.matchAll(rgx));
        const { dir } = path.parse(id);
        let newCode = code;
        for (const [all, m] of matches) {
          const fileName = path.relative(process.cwd(), path.resolve(dir, m))
            .replace('src/', '');
          const ref = this.emitFile({ type: 'asset', fileName, source: `contents of ${m}` });
          newCode = newCode.replace(all, 'import.meta.ROLLUP_FILE_URL_' + ref);
        }
        return {
          code: newCode,
          map: { mappings: '' },
        };
      }
    }
  ]
}

What does it do?

Well if you have this:

src
├── assets
│   ├── eye-closed.svg
│   └── eye-open.svg
├── atoms
│   └── cc-input-text.js

With a cc-input-text.js component using this to get the relative URLs:

const eyeClosedSvg = new URL('../assets/eye-closed.svg', import.meta.url).href;
const eyeOpenSvg = new URL('../assets/eye-open.svg', import.meta.url).href;

After a build, it would end up like this:

dist
├── my-bundle.js
├── eye-closed.svg
└── eye-open.svg

With my-bundle.js containing the code of cc-input-text.js and rewritten relative URLs like this:

const eyeClosedSvg = new URL('./eye-closed.svg', import.meta.url).href;
const eyeOpenSvg = new URL('./eye-open.svg', import.meta.url).href;

Improvements

  • It should be more agnostic to where your files are
  • It should be more agnostic to where your files go
  • It should allow you to hash your asset names and get your source code modified to reflect that
  • It will fail if a relative import is not found
  • No need for copy plugin anymore

TODO

  • Read the file instead of this fake content
  • Use Rollup's this.parse() instead of a regex
  • Rewrite the fileName so it's generic
  • Implement sourcemap
  • Investigate how it works if we hash JS files
  • Investigate how it works if we hash assets files
  • Add the classic include/exclude options
  • Proper filtering of other files in the transform hook (non js, non source...)

And of course:

  • a name
  • tests
  • docs

Questions / ideas

  1. What do we detect? .href or not at the end?
  2. If I want to optimize my SVG, should it be a transform option of this plugin or can this be done properly outside of this?
  3. Could rollup-plugin-visualizer be able to also display the size of my assets?
  4. Should this plugin focus only on this, or could it also rewrite the import.meta.url expression to something else?

About (4):

  • I think rewriting import.meta.url into something else that work in situations that don't understand this is a different problem.
  • I'm not a great fan of data URI inlining etc but maybe it could be an option...
@LarsDenBakker
Copy link

LarsDenBakker commented Jul 29, 2020

I think overall this idea is good. I don't agree with Jake's comment that these things "often end badly" or become brittle. You're going to be running this rollup plugin on your own code, not on your dependencies. So you know whether or not you've overwritten the global URL constructor, which you generally shouldn't be doing.

The AST parser could also check if URL is accessed from global, or being overwritten by another definition. In babel this is really easy, in Acorn it might be a bit more work. Can be an improvement after MVP. On the other hand, people could use just include/exclude to control this.

The transformation I think should look like this:

Input:

const myUrl = new URL('../assets/foo.png', import.meta.url);
const myHref = new URL('../assets/foo.png', import.meta.url).href;
const myPath = new URL('../assets/foo.png', import.meta.url).pathname;
const mySearchParams = new URL('../assets/foo.png', import.meta.url).searchParams;

Output:

const myUrl = new URL('./j4irjorj2o.png', import.meta.url);
const myHref = new URL('.j4irjorj2o.png', import.meta.url).href;
const myPath = new URL('./j4irjorj2o.png', import.meta.url).pathname;
const mySearchParams = new URL('./j4irjorj2o.png', import.meta.url).searchParams;

If I understand you correctly, rollup outputs new URL('./path/to/asset.ext', import.meta.url).href so this would actually become:

Output:

const myUrl = new URL(new URL('./j4irjorj2o.png', import.meta.url).href, import.meta.url);
const myHref = new URL(new URL('./j4irjorj2o.png', import.meta.url).href,  import.meta.url).href;
const myPath = new URL(new URL('./j4irjorj2o.png', import.meta.url).href,  import.meta.url).pathname;
const mySearchParams = new URL(new URL('./j4irjorj2o.png', import.meta.url).href,  import.meta.url).searchParams;

That's not a problem, the great thing about URL is that is composes. Just stick to transforming the first parameter of the URL constructor, and you won't need to worry about everything the user does to the URL.

One thing we need to be careful about here is usage of https://www.npmjs.com/package/babel-plugin-bundled-import-meta. It might conflict with this plugin, since this plugin outputs paths relative to the final bundle output while the bundled import meta plugin makes the paths relative to the original source location of the module.

We probably need to have an option for this to be able to handle both scenarios. We will probably use both of these plugins at ING at the same time.

Regarding transformations, I agree we should offer options for this in the plugin itself. Maybe even some good defaults too.

@hsablonniere
Copy link
Author

hsablonniere commented Jul 29, 2020

I think overall this idea is good. I don't agree with Jake's comment that these things "often end badly" or become brittle. You're going to be running this rollup plugin on your own code, not on your dependencies. So you know whether or not you've overwritten the global URL constructor, which you generally shouldn't be doing.

Right now, my use case is to run this on my component sources to prepare the npm package but when I will use my components in my other projects, I would need this plugin to run over my dependencies. I'm not using rollup on those projects so I'll need a similar trick for webpack 😄

The AST parser could also check if URL is accessed from global, or being overwritten by another definition. In babel this is really easy, in Acorn it might be a bit more work. Can be an improvement after MVP. On the other hand, people could use just include/exclude to control this.

I'll defer that to an improvement but yes it should be addressed.

If I understand you correctly, rollup outputs new URL('./path/to/asset.ext', import.meta.url).href so this would actually become:

I'm pretty sure we won't be able to ask Lukas to provide a config to remove the href so yes, maybe we need to compose it like your example. It's ugly but it works ™️

One thing we need to be careful about here is usage of npmjs.com/package/babel-plugin-bundled-import-meta. It might conflict with this plugin, since this plugin outputs paths relative to the final bundle output while the bundled import meta plugin makes the paths relative to the original source location of the module.

Good point, I'm using it in my webpack build to make my assets work.

Regarding transformations, I agree we should offer options for this in the plugin itself. Maybe even some good defaults too.

I have a transform callback in my POC and I wired svgo. Works great.

@hsablonniere
Copy link
Author

Next step: fork https://github.com/modernweb-dev/web and incubate/iterate here...

@jespertheend
Copy link

I made rollup-plugin-resolve-url-objects and this helped a lot, so thanks! It's still a bit rough around the edges but it works for new Worker() and such. I might add support for assets in the future, for now it only generates new chunks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment