Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Exclude node_modules in timemachine
find `pwd` -type d -maxdepth 3 -name 'node_modules' | xargs -n 1 tmutil addexclusion
@gianpaj

This comment has been minimized.

Copy link

@gianpaj gianpaj commented Nov 13, 2017

$ find `pwd` -maxdepth 3 -type d -name 'node_modules' | xargs -n 1 tmutil addexclusion

reording the -maxdepth option fixes this warning:

find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it).  Please specify options before other arguments.
@maxxcrawford

This comment has been minimized.

Copy link

@maxxcrawford maxxcrawford commented Jun 15, 2018

Can this command be run from the root user directory, or is there a specific directory?

@tomwidmer

This comment has been minimized.

Copy link

@tomwidmer tomwidmer commented Jun 21, 2018

To run it from a higher level if you have node projects in various subfolders, you can do:

find `pwd` -type d -maxdepth 8 -prune -name 'node_modules' | xargs -n 1 tmutil addexclusion

See also https://github.com/stevegrunwell/asimov

@gimdongwoo

This comment has been minimized.

Copy link

@gimdongwoo gimdongwoo commented Jul 9, 2018

I think, this is work.

find `pwd` -type d -maxdepth 8 -name 'node_modules' | xargs -n 1 tmutil addexclusion

What is -prune?

@fuerst

This comment has been minimized.

Copy link

@fuerst fuerst commented Aug 1, 2018

No need to pipe to xargs, just use find -exec like this:

find $(pwd) -type d -name node_modules -maxdepth 3 -exec tmutil addexclusion {} \;

Using $() for command substitution because back ticks are deprecated.

@osopolar

This comment has been minimized.

Copy link

@osopolar osopolar commented Aug 15, 2018

And how to exclude only the node_modules directory but not it's sub-directories for example only /Users/Example/Projects/test/node_modules but not /Users/Example/Projects/test/node_modules/write/node_modules? I guess it does not harm but it helps to keep the list cleaner.

@osopolar

This comment has been minimized.

Copy link

@osopolar osopolar commented Aug 15, 2018

You may use launchd to scan your project folder on a regular basis to exclude new node_modules directories, see https://gist.github.com/osopolar/9d8e8188091f7cbbfcee924ccfbb32dd

@qgy18

This comment has been minimized.

Copy link

@qgy18 qgy18 commented Aug 23, 2018

Thank you for your code, it helps a lot to me. At last, I use node.js to 'exclude only the node_modules directory but not it's sub-directories for example only'

/**
* exclude some directory from timemachine
* usage: 
* node find_node_modules.js | sudo xargs -n 1 tmutil addexclusion -p
* defail for -p flag: https://www.macworld.com/article/2033804/control-time-machine-from-the-command-line.html
*/

const dir = '/Users/your_name'; // base directory
const exclusionNames = ['node_modules', '.git']; 

const { readdirSync, statSync } = require('fs');
const { sep } = require('path');

function findNodeModules(dir, list) {
    list = list || [];

    readdirSync(dir).forEach(subDir => {
        let newDir = `${dir}${sep}${subDir}`;
        
        if(statSync(newDir).isDirectory()) {
            if(exclusionNames.includes(subDir.toLowerCase())) {
                list.push(newDir);
                return;
            }

            findNodeModules(newDir, list);
        }
    });
}

let list = [];
findNodeModules(dir, list);

console.log(list.join('\n'));
@giano

This comment has been minimized.

Copy link

@giano giano commented Aug 29, 2018

This tool is great for this: https://github.com/stevegrunwell/asimov

@dev01d

This comment has been minimized.

Copy link

@dev01d dev01d commented Feb 3, 2019

@gimdongwoo The -prune argument will prevent find from descending into the node_modules directory once found, which otherwise litters the exclude list. So if youre like me and have a ton of projects in your working directory, rather than trying to guess the depth in the file structure with -maxdepth you can use -prune to search the whole directory to add clean entries.

This version of @fuerst's command will get you only the top level node_modules directory of each occurrence and functions like adding them in the Time Machine GUI.

find $(pwd) -type d -name node_modules -prune -exec tmutil addexclusion {} \;
@techouse

This comment has been minimized.

Copy link

@techouse techouse commented May 9, 2019

I wrote a simple bash script which you can simply put in your development/work directory and run or add to your crontab.

#!/usr/bin/env bash

EXCLUDED_DIRECTORIES=( "env" "node_modules" "vendor" "venv" )

WORK_DIR=$(dirname "${BASH_SOURCE[0]}")
WORK_DIR=$(realpath "${WORK_DIR}")

for EXCLUDED_DIRECTORY in "${EXCLUDED_DIRECTORIES[@]}"; do
    find ${WORK_DIR} -maxdepth 2 -type d -name ${EXCLUDED_DIRECTORY} -prune -exec tmutil addexclusion {} \;
done
@chen86860

This comment has been minimized.

Copy link

@chen86860 chen86860 commented May 28, 2019

No need to pipe to xargs, just use find -exec like this:

find $(pwd) -type d -name node_modules -maxdepth 3 -exec tmutil addexclusion {} \;

Using $() for command substitution because back ticks are deprecated.

I cannot add exclude folders using your script but after some of arguments:

find $(pwd) -type d -name node_modules -maxdepth 3 -exec tmutil addexclusion -p {} \;

Thanks anyway.

@moekhalil

This comment has been minimized.

Copy link

@moekhalil moekhalil commented Jun 21, 2019

For some reason, prepending -prune would return an empty list.

Got this one for you guys/gals, working with prune and confirmation.

find `pwd` -name 'node_modules' -prune -type d -exec tmutil addexclusion {} \; -exec tmutil isexcluded {} \;

edit: The directories are showing as excluded when you run tmutil isexcluded <dir> (as the included command does) on each directory.

They do not however populate the list inside the table located at:

"System Preferences..." >> "Time Machine" Preferences >> "Options..." >> "Exclude these items from backup:".

I am almost certain that's expected and it should all work fine, but...SHOULD IT NOT WORK I will report back

@codexp

This comment has been minimized.

Copy link

@codexp codexp commented Jul 25, 2019

@giano thank you for the tool recommendation, it is great!

@glassdimly

This comment has been minimized.

Copy link

@glassdimly glassdimly commented Oct 5, 2019

Using this tool from @moekhalil, only some of the dirs confirm as excluded:

[Included]    /Users/jeremy/www/modules/node/test/fixtures/module-require/not-found/node_modules
[Included]    /Users/jeremy/www/modules/node/test/fixtures/module-require/parent/node_modules
[Included]    /Users/jeremy/www/modules/node/test/fixtures/module-require/child/node_modules
[Excluded]    /Users/jeremy/www/modules/node/deps/npm/node_modules
[Included]    /Users/jeremy/www/bric/node_modules
@glassdimly

This comment has been minimized.

Copy link

@glassdimly glassdimly commented Oct 5, 2019

I ended up just taking the list generated from the tool, cleaning it up and pasting its content into /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist.

They had to be formatted like so. All mine were relative to home dir, so I put them in that stanza:

<string>www/bric/node_modules</string>

Then I verified the exclusions are working:

$ tmutil isexcluded /Users/jeremy/www/bric/node_modules
[Excluded]    /Users/jeremy/www/bric/node_modules
$
@glassdimly

This comment has been minimized.

Copy link

@glassdimly glassdimly commented Oct 5, 2019

Alright, now here's a really good idea. Using .gitignore to generate excludes: https://github.com/samuelmeuli/tmignore

@moekhalil

This comment has been minimized.

Copy link

@moekhalil moekhalil commented Oct 5, 2019

@glassdimly glad you found something that worked. Running it locally, with those directories seemed like it worked for me. Are you sure you ran it in the most common parent (i.e. in Users/jeremy/www) or higher?

// modified just to spit out isexcluded
🛡  src/jeremy  ➜  find `pwd` -name 'node_modules' -prune -type d -exec tmutil isexcluded {} \;
[Included]    /src/jeremy/www/modules/node/test/fixtures/module-require/not-found/node_modules
[Included]    /src/jeremy/www/modules/node/test/fixtures/module-require/parent/node_modules
[Included]    /src/jeremy/www/modules/node/test/fixtures/module-require/child/node_modules
[Included]    /src/jeremy/www/modules/node/deps/npm/node_modules
[Included]    /src/jeremy/www/bric/node_modules
🛡  src/jeremy  ➜  find `pwd` -name 'node_modules' -prune -type d -exec tmutil addexclusion {} \; -exec tmutil isexcluded {} \;
[Excluded]    /src/jeremy/www/modules/node/test/fixtures/module-require/not-found/node_modules
[Excluded]    /src/jeremy/www/modules/node/test/fixtures/module-require/parent/node_modules
[Excluded]    /src/jeremy/www/modules/node/test/fixtures/module-require/child/node_modules
[Excluded]    /src/jeremy/www/modules/node/deps/npm/node_modules
[Excluded]    /src/jeremy/www/bric/node_modules
🛡  src/jeremy  ➜  find `pwd` -name 'node_modules' -prune -type d -exec tmutil isexcluded {} \;
[Excluded]    /src/jeremy/www/modules/node/test/fixtures/module-require/not-found/node_modules
[Excluded]    /src/jeremy/www/modules/node/test/fixtures/module-require/parent/node_modules
[Excluded]    /src/jeremy/www/modules/node/test/fixtures/module-require/child/node_modules
[Excluded]    /src/jeremy/www/modules/node/deps/npm/node_modules
[Excluded]    src/jeremy/www/bric/node_modules
@poma

This comment has been minimized.

Copy link

@poma poma commented Apr 7, 2020

Use this to wrap commands like npm or cargo to exclude any dependency dirs as you go. This prevents situations where Time Machine backs up your dependency dirs after you installed dependencies to a new project but before your regular search command is executed.

tmutil_exclude() {
    # todo: recurse to parent dirs to support commands that execute in project subdirs
    DIR=$1
    DEP_FILE=$2

    if [ -d "$DIR" ] && [ -f "$DEP_FILE" ] && ! tmutil isexcluded "$DIR" | grep -q '\[Excluded\]'; then
        tmutil addexclusion "$DIR"
        echo "tmutil: ${DIR} has been excluded from Time Machine backups"
    fi
}

__npm_wrapper () {
    command npm "$@"
    EXIT_CODE=$?
    tmutil_exclude "node_modules" "package.json"
    return $EXIT_CODE
}

alias npm=__npm_wrapper
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.