Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Lightning-fast project-wide find/replace with git grep and sed
gg_replace() {
if [[ "$#" == "0" ]]; then
echo 'Usage:'
echo ' gg_replace term replacement file_mask'
echo
echo 'Example:'
echo ' gg_replace cappuchino cappuccino *.html'
echo
else
find=$1; shift
replace=$1; shift
ORIG_GLOBIGNORE=$GLOBIGNORE
GLOBIGNORE=*.*
if [[ "$#" = "0" ]]; then
set -- ' ' $@
fi
while [[ "$#" -gt "0" ]]; do
for file in `git grep -l $find -- $1`; do
sed -e "s/$find/$replace/g" -i'' $file
done
shift
done
GLOBIGNORE=$ORIG_GLOBIGNORE
fi
}
gg_dasherize() {
gg_replace $1 `echo $1 | sed -e 's/_/-/g'` $2
}
@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 11, 2012

Thanks for this! Almost wrote it myself.

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 11, 2012

Awesome! Feedback (or diffs) welcome. I hope it's robust enough to work for somebody else.

One caveat: I find that in many cases I need to escape shell glob characters so my shell doesn't expand them. So instead of *.*, I need to use gg_replace this that \*\.\*

If you let the shell expand *.* or the like, you may just end up with way too many parameters for the script to run. In some cases, it may work fine though. I haven't analyzed it in a while.

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

What shell are you using?

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

Interesting. Looking at man sed:

NAME
sed -- stream editor

SYNOPSIS
sed [-Ealn] command [file ...]
sed [-Ealn] [-e command] [-f command_file] [-i extension] [file ...]

It looks like my sed command is written in the wrong order--but happens to work on Darwin. Let me change it to match the man file.

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

So the current version works better? I didn't get around to testing it locally until just now.

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

Just tried it, still giving me this error:

sed: can't read : No such file or directory
@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

Based on that, it sounds hard to write the command for both Darwin and Linux. There's gotta be a way, but the quick 'n' dirty solution sounds like taking out the '' for Linux.

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

in line 22, that is

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

right

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

You can see the cause for the problem if you compare the documentation for the -i parameter between Linux (http://linux.die.net/man/1/sed) and Mac OS X (https://developer.apple.com/library/mac/#documentation/Darwin/Reference/Manpages/man1/sed.1.html).

How annoying when Unix and Unix aren't the same. :\ I'd like to make it cross-platform, but don't have time at the moment.

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

sweet!

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

In OS X, the next argument is taken as the desired extension. If you wanted a backup, you'd use something like -i bak, and it would save the original of each alterred file as 'original.file.bak'. I passed it an empty string to fill the place of that argument, while telling it not to save a backup of the changed files.

By the way, I edited it yet again. I found that -i'' works fine on OS X. I suspect the same would work on Linux.

Thus continues the quest for platform cross-compatibility in one more little corner of the interwebz...

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

Oh, no, that's a different topic. The -i'' is hard-coded, and not affected by arguments you pass into gg_replace.

The third gg_replace argument (file mask) is for filtering what files you want to affect. So, if you only want to do a replacement in .html files, you can provide \*.html. Or you can leave out the mask, and affect every file in the repository.

Now I'm going to have to document it better. :)

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

Right, sed does no backup with -i''. I figured since this whole script does nothing outside a Git repo, sed's backup would be rather superfluous.

I do find it useful to run git diff after executing this. That let's me quickly scan for anything that's not what I really intended.

So glad it's working for you! Thanks for helping make it a bit more robust. Now when I try it from a Linux server, it should just work.

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

...and make it so SEO-friendly (or maybe that was happy accident)

right on. :)

This is where I keep it: https://github.com/jcamenisch/dotfiles/blob/master/.profile

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 12, 2012

Uh, yeah. I've read about the difference, but I'd have to look it up to remember, of course. The reason I use .profile is that I've moved to zsh, but want to use all the same stuff for times when I'm on bash, including on other machines like web servers. So I just call .profile from .bashrc and .zshrc.

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 12, 2012

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 13, 2012

Is if-shell the right way to do an OS conditional, like so?
ChrisJohnsen/tmux-MacOSX-pasteboard#8

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 13, 2012

That appears to be tmux-specific.

uname with no parameters is very consistent across Unix and Linux variants. It provides an OS name like "Darwin" or "Linux". I use that at https://github.com/jcamenisch/dotfiles/blob/master/.profile#L14 to include OS-specific configurations from a separate file (when it exists). This one command lets me provide custom config files targeting as many different operating systems as I want, although .profile_Darwin is the only one I actually have so far.

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 13, 2012

I guess the more direct way of doing that--with no external script file--would be with

if [[ `uname` = Linux ]]; then
  ...
fi

or for one-line shorthand:

[[ `uname` = Linux ]] && ...

(Warning: my sh syntax is likely to be wrong when I write something and don't test it.)

@zhangandyx

This comment has been minimized.

Copy link

@zhangandyx zhangandyx commented Jun 13, 2012

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 13, 2012

True. But you can link to a fork if you have a recommended change. Copy/paste isn't too much work with this amount of code.

@amueller

This comment has been minimized.

Copy link

@amueller amueller commented Jun 22, 2012

This is awesome. Thanks :)

@jcamenisch

This comment has been minimized.

Copy link
Owner Author

@jcamenisch jcamenisch commented Jun 22, 2012

Great to hear. Thanks!

@shadowhand

This comment has been minimized.

Copy link

@shadowhand shadowhand commented Jan 18, 2013

Here's a case-insensitive version:

gg_ireplace() {
  if [[ "$#" == "0" ]]; then
    echo 'Usage:'
    echo '  gg_ireplace term replacement file_mask'
    echo
    echo 'Example:'
    echo '  gg_ireplace cappuccino Cappuccino *.html'
    echo
  else
    find=$1; shift
    replace=$1; shift

    ORIG_GLOBIGNORE=$GLOBIGNORE
    GLOBIGNORE=*.*

    if [[ "$#" = "0" ]]; then
      set -- ' ' $@
    fi

    while [[ "$#" -gt "0" ]]; do
      for file in `git grep -Fil $find -- $1`; do
        perl -pi -w -e "s/\Q$find\E/$replace/gi;" $file
      done
      shift
    done

    GLOBIGNORE=$ORIG_GLOBIGNORE
  fi
}

Note the following, you may want to update your original script:

  • Perl used for the regex, which allows for case-insensitive option i
  • Perl has \Q...\E, which allows us to make the $find match literal
  • Usage of git grep -F for fixed-length string matching, makes initial search a bit faster

Also note that BSD sed command does not support the i option... I don't know if there is any way to do this without Perl.

@glasser

This comment has been minimized.

Copy link

@glasser glasser commented Jul 25, 2013

Odd, I get (on Mac) sed: -i may not be used with stdin. Changing -i'' to -i '' (with a space) fixes this, which makes sense: -i'' is exactly equivalent to -i, I think, so it's treating $file as the extension.

@dawalama

This comment has been minimized.

Copy link

@dawalama dawalama commented May 23, 2014

Here is my version that I use on Mac that handles filenames with 'Spaces'.

gg_replace() {
    if [[ "$#" == "0" ]]; then
        echo 'Usage:'
        echo '  gg_replace term replacement file_mask'
        echo
        echo 'Example:'
        echo '  gg_replace cappuchino cappuccino *.html'
        echo
    else
        find=$1; shift
        replace=$1; shift

        ORIG_GLOBIGNORE=$GLOBIGNORE
        GLOBIGNORE=*.*

        if [[ "$#" = "0" ]]; then
            set -- ' ' $@
        fi

        while [[ "$#" -gt "0" ]]; do
            for file in `git grep -l $find -- $1 | sed -e "s/ /~~~~~/g"`; do
                sed -i "" -e "s/$find/$replace/g" "${file/~~~~~/ }"
            done
            shift
        done

        GLOBIGNORE=$ORIG_GLOBIGNORE
    fi
}

@brunoro

This comment has been minimized.

Copy link

@brunoro brunoro commented Oct 7, 2015

just a small tip for those trying to use this on OS X: install GNU sed (brew install gnu-sed) and replace the sed call with gsed

@glyph

This comment has been minimized.

Copy link

@glyph glyph commented Oct 16, 2020

Here's my version: https://gist.github.com/glyph/9beafa8a7b26e5ca9f6666448fa5810d

gg_replace() {
    if [[ "$#" -lt "2" ]]; then
        echo "
Usage:
  $0 term replacement file_mask
Example:
  $0 cappuchino cappuccino '*.html'
";
    else
        local find="$1"; shift;
        local replace="$1"; shift;
        git grep -zlI "${find}" -- "$@" |
            xargs -0 sed -e "s/${find}/${replace}/g" -i '' ;
    fi;
}

A few notes:

  • passes shellcheck
  • doesn't depend on word splitting so doesn't need to monkey with GLOBIGNORE
  • supports spaces, newlines, and (apropos of dawalama's version ;-)) ~~~~~ in path names
  • it still has the problem described in dwheeler's https://dwheeler.com/essays/filenames-in-shell.html section 3.3, since I couldn't figure out a quick way to make git grep prefix all its output with ./
  • it's a bit faster since it doesn't spawn new sed processes quite so many times
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment