Lightning-fast project-wide find/replace with git grep and sed

.profile fragment

  gg_replace() {
    if [[ "$#" == "0" ]]; then
      echo 'Usage:'
      echo '  gg_replace term replacement file_mask'
      echo
      echo 'Example:'
      echo '  gg_replace cappuchino cappuccino *.html'
      echo
    else
      find=$1; shift
      replace=$1; shift

      ORIG_GLOBIGNORE=$GLOBIGNORE
      GLOBIGNORE=*.*
      
      if [[ "$#" = "0" ]]; then
        set -- ' ' $@
      fi

      while [[ "$#" -gt "0" ]]; do
        for file in `git grep -l $find -- $1`; do
          sed -e "s/$find/$replace/g" -i'' $file
        done
        shift
      done

      GLOBIGNORE=$ORIG_GLOBIGNORE
    fi
  }
  gg_dasherize() {
    gg_replace $1 `echo $1 | sed -e 's/_/-/g'` $2
  }

Oh I see, the ' ' is a default argument for -i? So i could do ``` gg_replace origstring replacestring ``` And it would do an in-place grep-replace without creating backups?

…

On Tue, Jun 12, 2012 at 1:16 PM, Jonathan Camenisch ***@***.*** wrote: In OS X, the next argument is taken as the desired extension. If you wanted a backup, you'd use something like `-i bak`, and it would save the original of each alterred file as 'original.file.bak'. I passed it an empty string to fill the place of that argument, while telling it not to save a backup of the changed files. By the way, I edited it yet again. I found that `-i''` works fine on OS X. I suspect the same would work on Linux. Thus continues the quest for platform cross-compatibility in one more little corner of the interwebz... --- Reply to this email directly or view it on GitHub: https://gist.github.com/1671995

Oh, no, that's a different topic. The -i'' is hard-coded, and not affected by arguments you pass into gg_replace.

The third gg_replace argument (file mask) is for filtering what files you want to affect. So, if you only want to do a replacement in .html files, you can provide \*.html. Or you can leave out the mask, and affect every file in the repository.

Now I'm going to have to document it better. :)

Makes sense; I did pass in *.rb, and it skipped the Rakefile. So gg_replace with -i'' means no backup possible? Worked great. Now I can rename my Rails apps and propagate that change throughout the structure :)

…

On Tue, Jun 12, 2012 at 1:29 PM, Jonathan Camenisch ***@***.*** wrote: Oh, no, that's a different topic. The `-i''` is hard-coded, and not affected by arguments you pass into gg_replace. The third argument (file mask) is for filtering what files you want to affect. So, if you only want to do a replacement in .html files, you can provide `\*.html`. Or you can leave out the mask, and affect every file in the repository. Now I'm going to have to document it better. :) --- Reply to this email directly or view it on GitHub: https://gist.github.com/1671995

Right, sed does no backup with -i''. I figured since this whole script does nothing outside a Git repo, sed's backup would be rather superfluous.

I do find it useful to run git diff after executing this. That let's me quickly scan for anything that's not what I really intended.

So glad it's working for you! Thanks for helping make it a bit more robust. Now when I try it from a Linux server, it should just work.

No problem, I'm glad somebody thought to make this, share it, and make it so SEO-friendly (or maybe that was happy accident). Yes, I don't see a point in doing backups inside a git repo either. Do you have this in a separate gg_replace.bash file, sourcing it from .bashrc or .bash_profile?

…

On Tue, Jun 12, 2012 at 1:48 PM, Jonathan Camenisch ***@***.*** wrote: Right, sed does no backup with `-i''`. I figured since this whole script does nothing outside a Git repo, sed's backup would be rather superfluous. I do find it useful to run `git diff` after executing this. That let's me quickly scan for anything that's not what I really intended. So glad it's working for you! Thanks for helping make it a bit more robust. Now when I try it from a Linux server, it should just work. --- Reply to this email directly or view it on GitHub: https://gist.github.com/1671995

...and make it so SEO-friendly (or maybe that was happy accident)

right on. :)

This is where I keep it: https://github.com/jcamenisch/dotfiles/blob/master/.profile

Yup, that looks like my bashrc. I started just putting everything in bashrc instead of bashprofile, but I don't have a solid understanding of the difference. I think one of them gets sourced on logon, as opposed to...?

…

On Tue, Jun 12, 2012 at 2:27 PM, Jonathan Camenisch < ***@***.*** > wrote: > ...and make it so SEO-friendly (or maybe that was happy accident) right on. :) This is where I keep it: https://github.com/jcamenisch/dotfiles/blob/master/.profile --- Reply to this email directly or view it on GitHub: https://gist.github.com/1671995

Uh, yeah. I've read about the difference, but I'd have to look it up to remember, of course. The reason I use .profile is that I've moved to zsh, but want to use all the same stuff for times when I'm on bash, including on other machines like web servers. So I just call .profile from .bashrc and .zshrc.

That's probably better than copying it over, which is what I did before I gave up on zsh.

…

On Tue, Jun 12, 2012 at 2:44 PM, Jonathan Camenisch < ***@***.*** > wrote: Uh, yeah. I've read about the difference, but I'd have to look it up to remember, of course. The reason I use .profile is that I've moved to zsh, but want to use all the same stuff for times when I'm on bash, including on other machines like web servers. So I just call .profile from [.bashrc](https://github.com/jcamenisch/dotfiles/blob/master/.bashrc) and [.zshrc](https://github.com/jcamenisch/dotfiles/blob/master/.zshrc). --- Reply to this email directly or view it on GitHub: https://gist.github.com/1671995

Is if-shell the right way to do an OS conditional, like so?
ChrisJohnsen/tmux-MacOSX-pasteboard#8

That appears to be tmux-specific.

uname with no parameters is very consistent across Unix and Linux variants. It provides an OS name like "Darwin" or "Linux". I use that at https://github.com/jcamenisch/dotfiles/blob/master/.profile#L14 to include OS-specific configurations from a separate file (when it exists). This one command lets me provide custom config files targeting as many different operating systems as I want, although .profile_Darwin is the only one I actually have so far.

I guess the more direct way of doing that--with no external script file--would be with

if [[ `uname` = Linux ]]; then
  ...
fi

or for one-line shorthand:

[[ `uname` = Linux ]] && ...

(Warning: my sh syntax is likely to be wrong when I write something and don't test it.)

Are there pull requests for gists? Looks like gists don't come with the same features as standard git repos.

…

On Tue, Jun 12, 2012 at 7:00 PM, Jonathan Camenisch < ***@***.*** > wrote: I guess the more direct way of doing that--with no external script file--would be with if [[ `uname` = Linux ]]; then ... fi or for one-line shorthand: [[ `uname` = Linux ]] && ... (Warning: my sh syntax is likely to be wrong when I write something and don't test it.) --- Reply to this email directly or view it on GitHub: https://gist.github.com/1671995

True. But you can link to a fork if you have a recommended change. Copy/paste isn't too much work with this amount of code.

This is awesome. Thanks :)

Great to hear. Thanks!

Here's a case-insensitive version:

gg_ireplace() {
  if [[ "$#" == "0" ]]; then
    echo 'Usage:'
    echo '  gg_ireplace term replacement file_mask'
    echo
    echo 'Example:'
    echo '  gg_ireplace cappuccino Cappuccino *.html'
    echo
  else
    find=$1; shift
    replace=$1; shift

    ORIG_GLOBIGNORE=$GLOBIGNORE
    GLOBIGNORE=*.*

    if [[ "$#" = "0" ]]; then
      set -- ' ' $@
    fi

    while [[ "$#" -gt "0" ]]; do
      for file in `git grep -Fil $find -- $1`; do
        perl -pi -w -e "s/\Q$find\E/$replace/gi;" $file
      done
      shift
    done

    GLOBIGNORE=$ORIG_GLOBIGNORE
  fi
}

Note the following, you may want to update your original script:

• Perl used for the regex, which allows for case-insensitive option i
• Perl has \Q...\E, which allows us to make the $find match literal
• Usage of git grep -F for fixed-length string matching, makes initial search a bit faster

Also note that BSD sed command does not support the i option... I don't know if there is any way to do this without Perl.

Odd, I get (on Mac) sed: -i may not be used with stdin. Changing -i'' to -i '' (with a space) fixes this, which makes sense: -i'' is exactly equivalent to -i, I think, so it's treating $file as the extension.

Here is my version that I use on Mac that handles filenames with 'Spaces'.

gg_replace() {
    if [[ "$#" == "0" ]]; then
        echo 'Usage:'
        echo '  gg_replace term replacement file_mask'
        echo
        echo 'Example:'
        echo '  gg_replace cappuchino cappuccino *.html'
        echo
    else
        find=$1; shift
        replace=$1; shift

        ORIG_GLOBIGNORE=$GLOBIGNORE
        GLOBIGNORE=*.*

        if [[ "$#" = "0" ]]; then
            set -- ' ' $@
        fi

        while [[ "$#" -gt "0" ]]; do
            for file in `git grep -l $find -- $1 | sed -e "s/ /~~~~~/g"`; do
                sed -i "" -e "s/$find/$replace/g" "${file/~~~~~/ }"
            done
            shift
        done

        GLOBIGNORE=$ORIG_GLOBIGNORE
    fi
}

just a small tip for those trying to use this on OS X: install GNU sed (brew install gnu-sed) and replace the sed call with gsed

Here's my version: https://gist.github.com/glyph/9beafa8a7b26e5ca9f6666448fa5810d

gg_replace() {
    if [[ "$#" -lt "2" ]]; then
        echo "
Usage:
  $0 term replacement file_mask
Example:
  $0 cappuchino cappuccino '*.html'
";
    else
        local find="$1"; shift;
        local replace="$1"; shift;
        git grep -zlI "${find}" -- "$@" |
            xargs -0 sed -e "s/${find}/${replace}/g" -i '' ;
    fi;
}

A few notes:

• passes shellcheck
• doesn't depend on word splitting so doesn't need to monkey with GLOBIGNORE
• supports spaces, newlines, and (apropos of dawalama's version ;-)) ~~~~~ in path names
• it still has the problem described in dwheeler's https://dwheeler.com/essays/filenames-in-shell.html section 3.3, since I couldn't figure out a quick way to make git grep prefix all its output with ./
• it's a bit faster since it doesn't spawn new sed processes quite so many times