Skip to content

Instantly share code, notes, and snippets.

@ShogunPanda
Created November 11, 2021 10:00
Show Gist options
  • Save ShogunPanda/752cce88659a09bff827ef8d2ecf8c80 to your computer and use it in GitHub Desktop.
Save ShogunPanda/752cce88659a09bff827ef8d2ecf8c80 to your computer and use it in GitHub Desktop.
const { dirname, sep, join, resolve } = require('path')
const { build } = require('esbuild')
const { readFile } = require('fs/promises')
// TODO: Check how to solve when [dir] or [hash] are used
function pinoPlugin(options) {
options = { transports: [], ...options }
return {
name: 'pino',
setup(currentBuild) {
const pino = dirname(require.resolve('pino'))
const threadStream = dirname(require.resolve('thread-stream'))
// Adjust entrypoints if it is an array
let entrypoints = currentBuild.initialOptions.entryPoints
if (Array.isArray(entrypoints)) {
let outbase = currentBuild.initialOptions.outbase
// Find the outbase
if (!outbase) {
const hierarchy = entrypoints[0].split(sep)
let i = 0
outbase = ''
let nextOutbase = ''
do {
outbase = nextOutbase
i++
nextOutbase = hierarchy.slice(0, i).join(sep)
} while (entrypoints.every(e => e.startsWith(`${nextOutbase}/`)))
}
const newEntrypoints = {}
for (const entrypoint of entrypoints) {
const destination = (outbase ? entrypoint.replace(`${outbase}/`, '') : entrypoint).replace(/.js$/, '')
newEntrypoints[destination] = entrypoint
}
entrypoints = newEntrypoints
}
// Now add our endpoints
const userEntrypoints = Object.entries(entrypoints)
const customEntrypoints = {
'thread-stream-worker': join(threadStream, 'lib/worker.js'),
'pino-worker': join(pino, 'lib/worker.js'),
'pino-pipeline-worker': join(pino, 'lib/worker-pipeline.js'),
'pino-file': join(pino, 'file.js')
}
// TODO: Add files in options.transport as well using require.resolve
currentBuild.initialOptions.entryPoints = { ...entrypoints, ...customEntrypoints }
// // Add a loader for all entrypoints to add the banner
currentBuild.onResolve({ filter: /\.js$/ }, args => {
if (args.kind === 'entry-point') {
const absolutePath = resolve(process.cwd(), args.path)
// Find in the entrypoints the one which has this definition in order to get the folder
const destination = userEntrypoints.find(pair => resolve(process.cwd(), pair[1]) === absolutePath)
if (destination) {
return { path: join(args.resolveDir, args.path), pluginData: { pinoBundlerOverride: destination[0] } }
}
}
return undefined
})
// Prepend our overrides
const banner = `/* Start of pino-webpack-bundler additions */
function pinoWebpackBundlerAbsolutePath(p) {
try {
return require('path').join(__dirname, p)
} catch(e) {
// This is needed not to trigger a warning if we try to use within CJS - Do we have another way?
const f = new Function('p', 'return new URL(p, import.meta.url).pathname');
return f(p)
}
}
`
currentBuild.onLoad({ filter: /\.js$/ }, async args => {
if (!args.pluginData || !args.pluginData.pinoBundlerOverride) {
return undefined
}
const contents = await readFile(args.path, 'utf8')
// Find how much the asset is nested
const prefix =
args.pluginData.pinoBundlerOverride
.split(sep)
.slice(0, -1)
.map(() => '..')
.join(sep) || '.'
const declarations = Object.keys(customEntrypoints)
.map(
id =>
`'${id === 'pino-file' ? 'pino/file' : id}': pinoWebpackBundlerAbsolutePath('${prefix}${sep}${id}.js')`
)
.join(',')
const overrides = `\nglobalThis.pinoBundlerOverrides = {${declarations}};\n/* End of pino-webpack-bundler additions */\n\n`
return {
contents: banner + overrides + contents
}
})
}
}
}
build({
entryPoints: {
main: 'src/index.js',
},
bundle: true,
platform: 'node',
outdir: 'dist',
plugins: [pinoPlugin({ transport: 'pino-pretty' })]
}).catch(() => process.exit(1))
@scorsi
Copy link

scorsi commented Mar 9, 2022

To finish the implementation of the transports add/change line 51:

const transportsEntrypoints = Object.fromEntries(options.transports.map((t) => [t, join(dirname(require.resolve(t)), 'index.js')]))
currentBuild.initialOptions.entryPoints = { ...entrypoints, ...customEntrypoints, ...transportsEntrypoints }

Also change the call of plugins to pinoPlugin({ transports: ['pino-pretty'] }).

But I still have issues with thread-stream trying to import lib/worker.js instead of thread-stream-worker.js. Any idea how to resolve that @ShogunPanda ?

EDIT1: I change filters from \.js$ to \.(js|ts)$ to load TypeScript files.

EDIT2: find that the reason is that the generated code is being added after Pino is being loaded so it has no effect, trying to figure out how to place the generated code before any of my ts file is being bundled.

@scorsi
Copy link

scorsi commented Mar 9, 2022

I finally found how to correctly implement that esbuild plugin, here's the code:

I add the overrides on the first pino.js file imports (I have two import of that lib (fastify dep + my own), instead of the entry point of the bundling. I use a global pinoBundlerRan variable to avoid running the code twice and if globalThis.__bundlerPathsOverrides is already defined I append the overrides to the object instead of redefining it.

const pinoPlugin = (options) => ({
    name: 'pino',
    setup(currentBuild) {
        const pino = dirname(require.resolve('pino'))
        const threadStream = dirname(require.resolve('thread-stream'))

        let entrypoints = currentBuild.initialOptions.entryPoints
        if (Array.isArray(entrypoints)) {
            let outbase = currentBuild.initialOptions.outbase

            if (!outbase) {
                const hierarchy = entrypoints[0].split(sep)

                let i = 0
                outbase = ''
                let nextOutbase = ''
                do {
                    outbase = nextOutbase
                    i++
                    nextOutbase = hierarchy.slice(0, i).join(sep)
                } while (entrypoints.every((e) => e.startsWith(`${nextOutbase}/`)))
            }

            const newEntrypoints = {}
            for (const entrypoint of entrypoints) {
                const destination = (outbase ? entrypoint.replace(`${outbase}/`, '') : entrypoint).replace(/.js$/, '')
                newEntrypoints[destination] = entrypoint
            }

            entrypoints = newEntrypoints
        }

        const customEntrypoints = {
            'thread-stream-worker': join(threadStream, 'lib/worker.js'),
            'pino-worker': join(pino, 'lib/worker.js'),
            'pino-pipeline-worker': join(pino, 'lib/worker-pipeline.js'),
            'pino-file': join(pino, 'file.js'),
        }
        const transportsEntrypoints = Object.fromEntries(
            (options.transports || []).map((t) => [t, join(dirname(require.resolve(t)), 'index.js')]),
        )
        currentBuild.initialOptions.entryPoints = { ...entrypoints, ...customEntrypoints, ...transportsEntrypoints }

        let pinoBundlerRan = false

        currentBuild.onEnd(() => { pinoBundlerRan = false })

        currentBuild.onLoad({ filter: /pino\.js$/ }, async (args) => {
            if (pinoBundlerRan) return
            pinoBundlerRan = true

            console.log(args.path)

            const contents = await readFile(args.path, 'utf8')

            const functionDeclaration = `
                    function pinoBundlerAbsolutePath(p) {
                        try {
                          return require('path').join(__dirname, p)
                        } catch(e) {
                          const f = new Function('p', 'return new URL(p, import.meta.url).pathname');
                          return f(p)
                        }
                      }`

            const pinoOverrides = Object.keys(customEntrypoints)
                .map((id) => `'${id === 'pino-file' ? 'pino/file' : id}': pinoBundlerAbsolutePath('./${id}.js')`)
                .join(',')

            const globalThisDeclaration = `globalThis.__bundlerPathsOverrides = 
                    globalThis.__bundlerPathsOverrides 
                        ? {...globalThis.__bundlerPathsOverrides, ${pinoOverrides}} 
                        : {${pinoOverrides}};`

            const code = functionDeclaration + globalThisDeclaration

            return {
                contents: code + contents,
            }
        })
    },
})

Thanks to @ShogunPanda for the first code which helps me a lot !

@ShogunPanda
Copy link
Author

Nice solution! Glad to have been helpful!

@wd-David
Copy link

I made a few changes to support different OS & typescript based on @scorsi's version:

  do {
    outbase = nextOutbase
    i++
    nextOutbase = hierarchy.slice(0, i).join(sep)
- } while (entrypoints.every((e) => e.startsWith(`${nextOutbase}/`)))
+ } while (entrypoints.every((e) => e.startsWith(`${nextOutbase}${sep}`)))
for (const entrypoint of entrypoints) {
- const destination = (outbase ? entrypoint.replace(`${outbase}/`, '') : entrypoint).replace(/.js$/, '')
+ const destination = (outbase ? entrypoint.replace(`${outbase}${sep}`, '') : entrypoint).replace(/.(js|ts)$/, '')
  newEntrypoints[destination] = entrypoint
}

Here is the code:

const pinoPlugin = (options) => ({
  name: 'pino',
  setup(currentBuild) {
    const pino = dirname(require.resolve('pino'))
    const threadStream = dirname(require.resolve('thread-stream'))

    let entrypoints = currentBuild.initialOptions.entryPoints
    if (Array.isArray(entrypoints)) {
      let outbase = currentBuild.initialOptions.outbase
      if (!outbase) {
        const hierarchy = entrypoints[0].split(sep)
        let i = 0
        outbase = ''
        let nextOutbase = ''
        do {
          outbase = nextOutbase
          i++
          nextOutbase = hierarchy.slice(0, i).join(sep)
        } while (entrypoints.every((e) => e.startsWith(`${nextOutbase}${sep}`)))
      }
      const newEntrypoints = {}
      for (const entrypoint of entrypoints) {
        const destination = (
          outbase ? entrypoint.replace(`${outbase}${sep}`, '') : entrypoint
        ).replace(/.(js|ts)$/, '')
        newEntrypoints[destination] = entrypoint
      }
      entrypoints = newEntrypoints
    }

    const customEntrypoints = {
      'thread-stream-worker': join(threadStream, 'lib/worker.js'),
      'pino-worker': join(pino, 'lib/worker.js'),
      'pino-pipeline-worker': join(pino, 'lib/worker-pipeline.js'),
      'pino-file': join(pino, 'file.js')
    }
    const transportsEntrypoints = Object.fromEntries(
      (options.transports || []).map((t) => [
        t,
        join(dirname(require.resolve(t)), 'index.js')
      ])
    )
    currentBuild.initialOptions.entryPoints = {
      ...entrypoints,
      ...customEntrypoints,
      ...transportsEntrypoints
    }

    let pinoBundlerRan = false

    currentBuild.onEnd(() => {
      pinoBundlerRan = false
    })

    currentBuild.onLoad({ filter: /pino\.js$/ }, async (args) => {
      if (pinoBundlerRan) return
      pinoBundlerRan = true

      const contents = await readFile(args.path, 'utf8')

      const functionDeclaration = `
        function pinoBundlerAbsolutePath(p) {
          try {
            return require('path').join(__dirname, p)
          } catch(e) {
            const f = new Function('p', 'return new URL(p, import.meta.url).pathname');
            return f(p)
          }
        }
      `

      const pinoOverrides = Object.keys(customEntrypoints)
        .map(
          (id) =>
            `'${
              id === 'pino-file' ? 'pino/file' : id
            }': pinoBundlerAbsolutePath('./${id}.js')`
        )
        .join(',')

      const globalThisDeclaration = `
        globalThis.__bundlerPathsOverrides =
          globalThis.__bundlerPathsOverrides
              ? {...globalThis.__bundlerPathsOverrides, ${pinoOverrides}}
              : {${pinoOverrides}};
      `

      const code = functionDeclaration + globalThisDeclaration

      return {
        contents: code + contents
      }
    })
  }
})

It works like a charm and kudos to @ShogunPanda & @scorsi!

Maybe it's great to have an esbuild plugin package just like pino-webpack-plugin.

@ShogunPanda
Copy link
Author

@davipon Looks amazing dude! Nice work!

Maybe it's great to have an esbuild plugin package just like pino-webpack-plugin.

That would be nice. Do you want to write one?

@wd-David
Copy link

Yeah, I'd like to write one.
While I'm relatively new to tooling unit tests, it might take some time 🙂.
I will update it here once it's ready.

Thanks again for your work @ShogunPanda.

@ShogunPanda
Copy link
Author

No problem sir! :)
Let me know when you have the package. I'll love to see it in action!

@wd-David
Copy link

Just released the first version of the plugin: esbuild-plugin-pino.
I learned so much from your work, and I appreciate that! 👏
It was my first time working on a plugin. Please kindly let me know if there are any problems or suggestions.

@ShogunPanda would you mind if I create a PR to add my plugin to this page? https://github.com/pinojs/pino/blob/master/docs/bundling.md

@ShogunPanda
Copy link
Author

ShogunPanda commented Jun 18, 2022

@davipon

You're welcome, your feedback made my day!
Thanks for the plugin, I will take a look soon.
Also, definitely create the PR, so we can list it.

CC: @mcollina

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment