rollup_bug.md

rollup/#1828

Vim/neovim moves a file before saving, and write its content into an another file (backupcopy option). Then, move it to original path. rollup uses fs.watch() for watching events such as changes and renames of a file. When fs.watch() notice an event, immediately rollup try to open a file by using fs.readFileSync(). Due to this behavior, fs.readFileSync() can't find a file when vim/neovim is used for editing it.

Source code:

• Using of fs.watch(): https://github.com/rollup/rollup/blob/5a7e1cb90dbc3d13359f3a141edab62f8b93e9d7/src/watch/fileWatchers.ts#L90
• Using of fs.readFileSync(): https://github.com/rollup/rollup/blob/5a7e1cb90dbc3d13359f3a141edab62f8b93e9d7/src/watch/fileWatchers.ts#L79

Some Webpages that mentions a related problem can be found:

• files - Using inotifywait along with vim - Unix & Linux Stack Exchange
• fs.watch() unreliable when using vim · Issue #3172 · nodejs/node-v0.x-archive
• Too many inotify events while editing in vim - Stack Overflow

There are some possible solutions:

1. Use of https://github.com/paulmillr/chokidar
2. Repeat trying to open a file until a saved file is moved to an original location.
3. Sleep a short time to wait for moving a file to original path.
4. Mix 1 and 2. Like this.

I guess that this problem relates to #1666,

4. node-fs.watch-for-vim.js
   • This script doesn't work expectedly in my environment.

There in an another problem. When a watching file is renamed by an editor, inotify, a mechanism fs.watch() uses in Linux, still watching a renamed file although the editor saves edited content to an another file. Because inotify watches a inode, not a filename.

Relates to #1619, #1669,

rollup/rollup-watch#16 rollup/rollup-watch#39

Summary

It seems that Vim/neovim moves a file before saving, and move it to original path after saving. Due to this behavior, fs.readFileSync() used by rollup can't find a file.

Environment

• Arch Linux 4.14 LTS
• node 10.1.0
• rollup 0.57.1
• Vim 8.0.1838 or neovim 0.2.2

How to reproduce

1. Execute npx --node-arg="--inspect" rollup --config --watch
2. In Vim/neovim, open and save a file rollup watches
3. rollup exits unexpectedly. exit status is 1.

How to get error messages

1. Execute npx --node-arg="--inspect" rollup --config --watch
2. Connect a debugger from chrome://inspect
3. In Vim/neovim, open and save a file rollup watches
4. Some errors are logged in a Chorme debug console

rollup:3919 Error: ENOENT: no such file or directory, open 'filename'
    at Object.fs.openSync (fs.js:559:3)
    at Object.fs.readFileSync (fs.js:465:33)
    at FSWatcher.handleWatchEvent (path_to_project/node_modules/rollup/dist/rollup.js:23360:35)

rollup --watch uses fs.watch(). I wrote a short script to see what events are emitted by inotify.

When save a file by using a Vim/neovim, output is:

start watching
eventType: rename, file: 4913
ls: cannot access 'test.mjs': No such file or directory
eventType: change, file: 4913
ls: cannot access 'test.mjs': No such file or directory
eventType: rename, file: 4913
ls: cannot access 'test.mjs': No such file or directory
eventType: rename, file: test.mjs
ls: cannot access 'test.mjs': No such file or directory
eventType: rename, file: test.mjs
-rw-r--r-- 1 syusui syusui 360 May 25 02:10 test.mjs

eventType: change, file: test.mjs
-rw-r--r-- 1 syusui syusui 360 May 25 02:10 test.mjs

eventType: change, file: test.mjs
-rw-r--r-- 1 syusui syusui 360 May 25 02:10 test.mjs

When save a file by using nano, output is:

start watching
eventType: change, file: test.mjs
-rw-r--r-- 1 syusui syusui 360 May 25 02:12 test.mjs

eventType: change, file: test.mjs
-rw-r--r-- 1 syusui syusui 360 May 25 02:12 test.mjs

A differences is that vim showed eventType: rename, but nano didn't show it.

According to a reference document:

Note that on most platforms, 'rename' is emitted whenever a filename appears or disappears in the directory.

It seems that Vim (or some other editors) moves a file before saving, and move it to original path after saving.

Related issues can be found:

• Too many inotify events while editing in vim - Stack Overflow
• fs.watch() unreliable when using vim · Issue #3172 · nodejs/node-v0.x-archive

Reason

Vim (or some other editors) moves a file before saving, and move it to original path after saving.

How to solve this problem

Short workaround

workaround.js

const fs = require('fs');

/** watch for safe-write
 * @param {string}   initialFilename
 * @param {number}   ignoreTime      milli-seconds
 * @param {function} handler
 */
const watch = (initialFilename, ignoreTime, handler) => {
  let lastChange = Date.now();

  const watcher = fs.watch(initialFilename, (ev, filename) => {
    // console.warn("[DEBUG]", ev, filename);
    switch (ev) {
    case 'rename':
      watcher.close();
      try {
        // Try to open **filename** when renamed
        watch(filename, ignoreTime, handler);
      } catch (err) {
        if (err.code === 'ENOENT') {
          // File is deemed to be overwritten safely (Vim backupcopy)
          // https://github.com/nodejs/node-v0.x-archive/issues/3172
          // https://github.com/rollup/rollup-watch/issues/4
          //
          // if fs.watch failed, exception will be thrown
          watch(initialFilename, ignoreTime, handler);
        } else {
          // Re-throw error if unknown error
          throw err;
        }
      }
      break;
    case 'change':
      {
        const now = Date.now();
        // console.log("[DEBUG]", lastChange);
        if ((now - lastChange) >= ignoreTime) {
          lastChange = now;
          handler(ev, filename);
        }
      }
      break;
    }
  });
};


const file = 'foobar.js';
watch(file, 400, console.log);