Skip to content

Instantly share code, notes, and snippets.

@hsablonniere
Last active Nov 15, 2021
Embed
What would you like to do?
Explainer for my rollup plugin idea about import.meta.url with assets

Explainer for my rollup plugin idea

The "i-prefer-import-meta-url-over-using-esm-import-to-get-a-relative-file-url" plugin ;-)

Context

In my components library, I have SVG images. I used them with <img> tags in my components and to get the proper relative URL, I use new URL('../assets/image.svg', import.meta.url').href.

To make this work in a rollup build, you need to:

  • Preserve relative file tree structure of your JavaScript files
  • Copy files with rollup-plugin-copy and keep same relative tree structure

I had to rework my tree structure to have all components in src/foo/component-foo.js and all assets in src/assets/image.svg. I also had to make sure my components end up in src/dist/component-foo.js and copied assets in dist/assets/image.svg.

It works "OK" in my situation but it has a few limitations:

  • The rules on the tree structure are too strict
  • When I did a test to emit just one bundle instead of preserving modules, I had to emit it in a subdir so the ../assets still work.
  • If a relative path does not exist, the build does not fail.
  • You need the copy plugin.

Why don't you use ESM import for your assets?

"Everyone is doing it", especially with Webpack loaders but it's not standard. It requires specific bundler config to work before serving the source files to a browser.

With import.meta.url it works as is in the browser before the bundling. This plugin idea is "just a way" to help rollup understand this pattern better and rewrite stuffs if the relative tree structure is changed.

A solution

A tranform plugin (with transform() hook) that:

  • detects new URL('../path/to.svg', import.meta.url)
  • emits file with this.emitFile({type:'asset', /* .... */})
  • replaces the new URL('../path/to.svg', import.meta.url) with import.meta.ROLLUP_FILE_URL_referenceId so rollup can replace it with the correct relative path (and maybe the hash).

Proof of concept

Here's the untested/dirty/hacky proof of concept:

// rollup.config.js

export default {
  // ...
  plugins: [
    // ...
    {
      transform(code, id) {
        const rgx = /new URL\('(.*)', import\.meta\.url\)\.href/g;
        const matches = Array.from(code.matchAll(rgx));
        const { dir } = path.parse(id);
        let newCode = code;
        for (const [all, m] of matches) {
          const fileName = path.relative(process.cwd(), path.resolve(dir, m))
            .replace('src/', '');
          const ref = this.emitFile({ type: 'asset', fileName, source: `contents of ${m}` });
          newCode = newCode.replace(all, 'import.meta.ROLLUP_FILE_URL_' + ref);
        }
        return {
          code: newCode,
          map: { mappings: '' },
        };
      }
    }
  ]
}

What does it do?

Well if you have this:

src
├── assets
│   ├── eye-closed.svg
│   └── eye-open.svg
├── atoms
│   └── cc-input-text.js

With a cc-input-text.js component using this to get the relative URLs:

const eyeClosedSvg = new URL('../assets/eye-closed.svg', import.meta.url).href;
const eyeOpenSvg = new URL('../assets/eye-open.svg', import.meta.url).href;

After a build, it would end up like this:

dist
├── my-bundle.js
├── eye-closed.svg
└── eye-open.svg

With my-bundle.js containing the code of cc-input-text.js and rewritten relative URLs like this:

const eyeClosedSvg = new URL('./eye-closed.svg', import.meta.url).href;
const eyeOpenSvg = new URL('./eye-open.svg', import.meta.url).href;

Improvements

  • It should be more agnostic to where your files are
  • It should be more agnostic to where your files go
  • It should allow you to hash your asset names and get your source code modified to reflect that
  • It will fail if a relative import is not found
  • No need for copy plugin anymore

TODO

  • Read the file instead of this fake content
  • Use Rollup's this.parse() instead of a regex
  • Rewrite the fileName so it's generic
  • Implement sourcemap
  • Investigate how it works if we hash JS files
  • Investigate how it works if we hash assets files
  • Add the classic include/exclude options
  • Proper filtering of other files in the transform hook (non js, non source...)

And of course:

  • a name
  • tests
  • docs

Questions / ideas

  1. What do we detect? .href or not at the end?
  2. If I want to optimize my SVG, should it be a transform option of this plugin or can this be done properly outside of this?
  3. Could rollup-plugin-visualizer be able to also display the size of my assets?
  4. Should this plugin focus only on this, or could it also rewrite the import.meta.url expression to something else?

About (4):

  • I think rewriting import.meta.url into something else that work in situations that don't understand this is a different problem.
  • I'm not a great fan of data URI inlining etc but maybe it could be an option...
@hsablonniere
Copy link
Author

hsablonniere commented Jul 24, 2020

About import.meta.resolve...

The new URL('../assets/image.svg, import.meta.url) looks a lot like the import.meta.resolve('../asset/image.svg') proposition/draft but what is it anyway?

@hsablonniere
Copy link
Author

hsablonniere commented Jul 24, 2020

Feedbacks from @jakearchibald

Nice! Although that breaks sourcemaps right? This is why I stick to static imports for this stuff. I don't want to have to mess with an AST.

Fair enough.

Also, like I said in the video, I prefer my build magic to be obvious. I don't really like plugins that take some valid JS and make it behave differently.

I prefer my build to take valid JS that works without a bundler.
Which means I don't want an ESM import of a non JS module to retrieve a URL.

I guess it's a matter of opinion, taste and choice on this one.

Rollup does AST magic with static imports and dynamic imports. You can't override/alias import in your code, but you can override/alias URL. Either your plugin will deal with this, and be massively complicated, or it won't, and it'll be fragile.

That's why I don't like plugins that do magic with new Worker(…), since it often ends badly

That's 100% and it's a very interesting feedback.
In the end, we're not really talking about removing the new URL() or the import.meta.url but "just" updating the relative path (first arg) of new URL() so I'm not that worried.

@hsablonniere
Copy link
Author

hsablonniere commented Jul 24, 2020

Feedbacks from @lukastaegert

https://twitter.com/lukastaegert/status/1286676039469588480

Sounds like a cool idea! One thought from skimming through the readme: Using a regexp to scan for import.meta may be fast but also brittle. Using this.parse is slower but gives you the AST. If you return the ast with the code you can make up for it if your plugin is last

https://twitter.com/lukastaegert/status/1286684134803492866

When you parse the ast in a transform hook and do not modify the code, instead of just the code you can return a {code,ast} object. This can help Rollup prevent a double parse of the same code. If you modify the code it does not matter, though.

@hsablonniere
Copy link
Author

hsablonniere commented Jul 24, 2020

About our questions:

  1. What do we detect? .href or not at the end?

The import.meta.ROLLUP_FILE_URL_ system outputs new URL('./path/to/asset.ext', import.meta.url).href so I guess we should also match exactly with .href and maybe consider a boolean option for users that rely on auto toString() of the URL object.

  1. If I want to optimize my SVG, should it be a transform option of this plugin or can this be done properly outside of this?

Looking at the list of plugins in https://github.com/rollup/awesome, it seems like there are no way to transform assets.
Lots of plugins do it with the import mechanism like https://github.com/porsager/rollup-plugin-svgo.

If you look at https://github.com/vladshcherbin/rollup-plugin-copy, it provides a transform option where you could use image optimizer etc like svgo. We should do the same.

  1. Could rollup-plugin-visualizer be able to also display the size of my assets?

Not sure yet...

  1. Should this plugin focus only on this, or could it also rewrite the import.meta.url expression to something else?

I already answered this.

@hsablonniere
Copy link
Author

hsablonniere commented Jul 24, 2020

Capture d’écran_2020-07-25_01-08-50

🎉 Yeah for detailed errors when doing mistakes with relative assets and import.meta.url

@LarsDenBakker
Copy link

LarsDenBakker commented Jul 29, 2020

I think overall this idea is good. I don't agree with Jake's comment that these things "often end badly" or become brittle. You're going to be running this rollup plugin on your own code, not on your dependencies. So you know whether or not you've overwritten the global URL constructor, which you generally shouldn't be doing.

The AST parser could also check if URL is accessed from global, or being overwritten by another definition. In babel this is really easy, in Acorn it might be a bit more work. Can be an improvement after MVP. On the other hand, people could use just include/exclude to control this.

The transformation I think should look like this:

Input:

const myUrl = new URL('../assets/foo.png', import.meta.url);
const myHref = new URL('../assets/foo.png', import.meta.url).href;
const myPath = new URL('../assets/foo.png', import.meta.url).pathname;
const mySearchParams = new URL('../assets/foo.png', import.meta.url).searchParams;

Output:

const myUrl = new URL('./j4irjorj2o.png', import.meta.url);
const myHref = new URL('.j4irjorj2o.png', import.meta.url).href;
const myPath = new URL('./j4irjorj2o.png', import.meta.url).pathname;
const mySearchParams = new URL('./j4irjorj2o.png', import.meta.url).searchParams;

If I understand you correctly, rollup outputs new URL('./path/to/asset.ext', import.meta.url).href so this would actually become:

Output:

const myUrl = new URL(new URL('./j4irjorj2o.png', import.meta.url).href, import.meta.url);
const myHref = new URL(new URL('./j4irjorj2o.png', import.meta.url).href,  import.meta.url).href;
const myPath = new URL(new URL('./j4irjorj2o.png', import.meta.url).href,  import.meta.url).pathname;
const mySearchParams = new URL(new URL('./j4irjorj2o.png', import.meta.url).href,  import.meta.url).searchParams;

That's not a problem, the great thing about URL is that is composes. Just stick to transforming the first parameter of the URL constructor, and you won't need to worry about everything the user does to the URL.

One thing we need to be careful about here is usage of https://www.npmjs.com/package/babel-plugin-bundled-import-meta. It might conflict with this plugin, since this plugin outputs paths relative to the final bundle output while the bundled import meta plugin makes the paths relative to the original source location of the module.

We probably need to have an option for this to be able to handle both scenarios. We will probably use both of these plugins at ING at the same time.

Regarding transformations, I agree we should offer options for this in the plugin itself. Maybe even some good defaults too.

@hsablonniere
Copy link
Author

hsablonniere commented Jul 29, 2020

I think overall this idea is good. I don't agree with Jake's comment that these things "often end badly" or become brittle. You're going to be running this rollup plugin on your own code, not on your dependencies. So you know whether or not you've overwritten the global URL constructor, which you generally shouldn't be doing.

Right now, my use case is to run this on my component sources to prepare the npm package but when I will use my components in my other projects, I would need this plugin to run over my dependencies. I'm not using rollup on those projects so I'll need a similar trick for webpack 😄

The AST parser could also check if URL is accessed from global, or being overwritten by another definition. In babel this is really easy, in Acorn it might be a bit more work. Can be an improvement after MVP. On the other hand, people could use just include/exclude to control this.

I'll defer that to an improvement but yes it should be addressed.

If I understand you correctly, rollup outputs new URL('./path/to/asset.ext', import.meta.url).href so this would actually become:

I'm pretty sure we won't be able to ask Lukas to provide a config to remove the href so yes, maybe we need to compose it like your example. It's ugly but it works ™️

One thing we need to be careful about here is usage of npmjs.com/package/babel-plugin-bundled-import-meta. It might conflict with this plugin, since this plugin outputs paths relative to the final bundle output while the bundled import meta plugin makes the paths relative to the original source location of the module.

Good point, I'm using it in my webpack build to make my assets work.

Regarding transformations, I agree we should offer options for this in the plugin itself. Maybe even some good defaults too.

I have a transform callback in my POC and I wired svgo. Works great.

@hsablonniere
Copy link
Author

hsablonniere commented Jul 29, 2020

Next step: fork https://github.com/modernweb-dev/web and incubate/iterate here...

@jespertheend
Copy link

jespertheend commented Nov 15, 2021

I made rollup-plugin-resolve-url-objects and this helped a lot, so thanks! It's still a bit rough around the edges but it works for new Worker() and such. I might add support for assets in the future, for now it only generates new chunks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment