Webpack funland configuration

emit.png

wpCacheBusting.md

I've been trying to cache bust our assets using webpack for quite some time now, and so I wanted to discuss some of the issues we've been running into consistently when trying to close out this ticket.

Namely, our use of postcss-loader and how that effects the path resolution of our static assets (especially in conjunction with file-loader). I think the best way to talk about these issues is just to give a brief overview, and then discuss patterns attempted and why they've failed.

base.js

• This file's primary function is to bring in our asset loaders and resolve files for babel. Here's a configuration I'm using to hash images/fonts when they are imported. There is a condition to preserve our use of raw svgs for font awesome:

module: {
    rules: [
      {
        test: /\.(otf|svg|ttf|png|gif|jpeg|jpg|eot|woff2?)(\?v=\d+\.\d+\.\d+)?$/,
        exclude: [
          resolve(__dirname, '../assets/raw-svg/fontawesome')
        ],
        use: {
          loader: 'file-loader',
          options: {
            name: '[path][name].[ext]?[hash]',
            path: './'
          }
        }
      },
      {
        test: /\.svg$/,
        include: [
          resolve(__dirname, '../assets/raw-svg/fontawesome')
        ],
        use: {
          loader: 'raw-loader'
        }
      },

• On its own, this snippet sings in conjunction with our image object that handles all of our imports (see PR#2090). It obeys our rules to exclude any svgs inside of raw-svg/fontawesome and can be extended to exclude the entire parent directory if need be.

• Take note that the names of the files now include the relative path of the file. And that the publicPath is not defined.

• The rest of the file deals with babel which we don't really need to get into here.

client_bundle.js

• My favorite file, and as the name suggests, deals with the client-side bundle. At the top of the file more babel rules are being defined, but the fun part starts when we configure our source maps and configure postcss-loader plus other loaders for our stylesheets:

entry: {
    styles: join(__dirname, '../css/app'),
    ads: join(__dirname, '../js/ads'),
    scripts: [
      'babel-polyfill',
      join(__dirname, '../js/client')
    ]
  },
  output: {
    path: join(__dirname, '../public/assets'),
    publicPath: '/assets/',
    filename: '[name].[chunkhash].js',
    chunkFilename: '[name].[chunkhash].js',
    sourceMapFilename: '[name].[chunkhash].js.map'
  },
  devtool: '#source-map',
    module: {
    rules: [
      ...baseRules,
      {
        test: /\.scss$/,
        use: ExtractTextPlugin.extract({
          fallback: 'style-loader',
          use: [
            {
              loader: 'css-loader',
              options: {
                minimize: true
              }
            },
            {
              loader: 'postcss-loader',
              options: {
                plugins: () => [
                  autoprefixer
                ]
              }
            },
            'sass-loader'
          ]
        })
      }
    ]
  },

• Without having to go line by line, what's important is that we are generating our source maps to ./rosetta/public/assets, and we have this public path /assets/ . The CSS configurations are pretty self evident, besides the fact ExtractTextPlugin is being used to generate a new CSS file.

dev_server.js

• This file has similar babel logic but what's important here is that it too has it's own CSS configuration.

export default {
  ...baseConfig,
  module: {
    ...baseConfig.module,
    rules: [
      ...baseRules,
      {
        test: /\.scss$/,
        loaders: [
          'style-loader',
          'css-loader',
          {
            loader: 'postcss-loader',
            options: {
              plugins: () => [
                autoprefixer
              ]
            }
          },
          'sass-loader'
        ]
      }
    ]
  },
  entry: [
    'babel-polyfill',
    join(__dirname, '../js/ads'),
    join(__dirname, '../js/client'),
    join(__dirname, '../css/app'),
    'webpack/hot/dev-server',
    'webpack-dev-server/client?http://localhost:8080'
  ],
  output: {
    path: __dirname,
    publicPath: '/',
    filename: 'bundle.js'
  },

karma.js && pm2_server_bundle.js don't seem to effect this ticket in any way and do not have their own unique configurations for handling assets. I welcome everyone to take a look at those files in case I've missed something that could solve this issue!

Attempted Solutions/Issues

The reason I didn't present the default configuration of base.js but instead showed the desired implementation is to illustrate the hierarchy of configs and to show that all of our static assets (excluding raw-svgs) will use file-loader.

Here is a solution that should work on your local machine if you run gulp prod-dev or gulp dev:

client_bundle.js

export default {
  ...baseConfig,
  entry: {
    styles: join(__dirname, '../css/app'),
    ads: join(__dirname, '../js/ads'),
    scripts: [
      'babel-polyfill',
      join(__dirname, '../js/client')
    ]
  },
  output: {
    path: join(__dirname, '../public/assets'),
    publicPath: '/assets/',
    filename: '[name].[chunkhash].js',
    chunkFilename: '[name].[chunkhash].js',
    sourceMapFilename: '[name].[chunkhash].js.map'
  },
  devtool: '#source-map',
  module: {
    rules: [
      ...baseRules,
      {
        test: /\.scss$/,
        use: ExtractTextPlugin.extract({
          fallback: 'style-loader',
          use: [
            {
              loader: 'css-loader',
              options: {
                sourceMap: true
              }
            },
            {
              loader: 'postcss-loader',
              options: {
                plugins: () => [
                  autoprefixer,
                  postcssImport,
                  url({
                    url: 'copy',
                    // base path to search assets from
                    basePath: join(__dirname, '../'),
                    // dir to copy assets
                    assetsPath: '../../public/assets',
                    // using hash names for assets (generates from asset content)
                    useHash: true
                  })
                ]
              }
            },
            'sass-loader?sourceMap'
          ]
        })
      }
    ]
  },
...

As you can see every loader has a sourceMap and that is a requirement for postcss to work correctly when hashing or doing imports. If all things run well you should see this type of emission in your console:

Things seem to work as they are suppose to, but this will fail on any Openstack instance after npm run build...

On any given Openstack instance, /srv/rosetta-deploys/ denotes the base directory of the repo. And it's attempting to write public to the parent directory of said repo, which errors out: Module build failed: Error: EACCES: permission denied, mkdir '/srv/public'

And rightly so...but what's baffling is that this behavior is inconsistent with the local environment i.e. public is not being written to the parent directory of /rosetta. Also we have this _ directory prefixing the path that is not explicitly defined.

We are really close to the expected behavior we want to achieve: static assets in the assets folder, when referenced in the code either via CSS or JS import, copy's itself to a directory in public/assets with a hashed filename. I've tried many many many iterations of path configurations abusing join and resolve and things are still failing hard or only work locally.

I'm scratching my head about what's wrong at this point and wanted to open it up to others who may know more about our build process / have suggestions for different Webpack configurations. Any ideas at all are greatly appreciated! If anything here is unclear or needs more explanation please reach out...looking for a friend.