Instant grep + quickfix

grep.md

FOREWORDS

I don't mean the snippet at the bottom of this gist to be a generic plug-n-play solution to your search needs. It is very likely to not work for you or even break things, and it certainly is not as extensively tested and genericised as your regular third-party plugin.

My goal, here and in most of my posts, is to show how Vim's features can be leveraged to build your own high-level, low-maintenance, workflows without systematically jumping on the plugins bandwagon or twisting Vim's arm.

---

Instant grep + quickfix

:help :grep is a fantastic tool, especially when combined with the quickfix, but it is certainly a bit rough around the edges. Let's see what it takes to smooth it out.

Tell Vim what external command to use for grepping

By default, Vim is set to use grep for :grep, which seems logical given the name of the command and all. But it also lets us customize the actual command via :help 'grepprg'. We can use it to add default parameters, for example, or to point Vim to a completely different program.

Over the years, people have been busy working on faster and/or more developer-friendly grep alternatives. The first such effort that I remember of was ack, then ag changed the rules of the game and other players entered the field, the latest being rg. Depending on your use cases, switching :grep to one of those alternatives may be a wise choice.

In the example below I use ag but feel free to substitute it with whatever you want.

set grepprg=ag\ --vimgrep

Now, this change is not strictly necessary for what we are trying to do but, even if we stopped here, using ag or rg over grep can be a cheap and non-negligible upgrade to the built-in :grep and :lgrep.

Perform the search in a sub-shell

One of those rough edges I alluded to earlier is the fact that Vim executes the command in Vim's parent shell. This has a few consequences that contribute to a less than ideal experience:

• Vim is suspended for how long it takes the command to finish.
• The results are printed in the shell and clog our terminal's scroll buffer.
• We must press the dreaded <CR> to come back to Vim.

Enters a sister command of :grep, :help :cgetexpr, that takes an "expression" as input. In concrete terms, :cgetexpr takes the raw output of grep or ag or whatever, parses it according to :help 'errorformat' and creates a quickfix list. :cgetexpr operates at a lower level than :grep and doesn't involve any I/O so it is simpler and faster.

:cgetexpr <expression>

Now, how do we get the output of ag if we can't execute it in the shell? Via :help system() of course:

:cgetexpr system('ag foo')

But that's a lot to type, right? Well, how about wrapping it in a custom function?

function! Grep(args)
    return system('ag ' . a:args)
endfunction

Now we have the following, which seems to solve all of the issues outlined above:

:cgetexpr Grep('foo')

But we already have ag set as our 'grepprg' so we might as well use that, making our function future-proof:

function! Grep(args)
    return system(&grepprg . ' ' . a:args)
endfunction

From there, we can modify the function to handle multiple arguments because :grep 'foo bar' filename is a very reasonable thing to do:

function! Grep(...)
    return system(&grepprg . ' ' . join(a:000, ' '))
endfunction

There are many ways to write the expression above, and choosing one or the other is mostly a matter of personal preference… and performance:

Expression	Performance	Stylish
&grepprg . ' ' . join(a:000, ' ')	0.000072s	No
join(extend([&grepprg], a:000), ' ')	0.000065s	Yes
join([&grepprg] + a:000, ' ')	0.000050s	Yes

The third one it is, then:

function! Grep(...)
    return system(join([&grepprg] + a:000, ' '))
endfunction

What this does is:

• concatenate a list with &grepprg as sole value with a list containing all the arguments passed to the function,
• join the resulting list into a single string,
• pass it to system().

Nice… Our little function works very well but calling it manually kind of defeats the point we are trying to make.

Create a command to populate the quickfix list

We will start small and simple with a simple :Grep command that takes one or more arguments:

command! -nargs=+ Grep cgetexpr grep(<f-args>)

which seems to do the job:

:Grep foo

but we might want to restrict our search to a given directory or file so it might be a good idea to use command-line completion:

command! -nargs=+ -complete=file_in_path Grep cgetexpr Grep(<f-args>)

and we might want to execute another command after :Grep foo, so we should add another flag:

command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)

and let's not forget :help :lgrep:

command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)

Aaaaand we are done with the usability issues mentioned above. Here is our little snippet in all its glory:

function! Grep(...)
    return system(join([&grepprg] + a:000), ' '))
endfunction
command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)

Dealing with not-so-edge case

Our custom :Grep command seems to do a good job: :Grep foo *.js is faster than ever and we don't have to press <Enter> anymore. This is all good but :grep expands wildcards by default (think :grep foo % or :grep bar ##) and we don't want to lose that, so we will have to modify our expression slightly:

return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))

Here we use :help expandcmd() to expand any glob present in the "arguments" part of the command before it is executed, which makes :Grep foo % work like :grep foo %.

Neat!

Open the location/quickfix window automatically if there are valid entries in the list

There is one usability issue that we haven't mentioned before: when the search is done, we get a little message about the first item in our search but that's all. True, we can use :help :cn and friends to jump around quickfix entries but, if we want to see the results, we have to use :help :cwindow:

:Grep foo
:cwindow

or, shorter:

:Gr foo | cw

What if we could open the quickfix window automatically when the quickfix list changes?

Autocommands to the rescue! The first step is to define a self-clearing group so that our autocommands don't needlessly pile up if we re-:source our vimrc:

augroup quickfix
    autocmd!
    " our autocommands here
augroup END

Then, we add two autocommands:

• one to execute :cwindow whenever :cgetexpr is executed,
• another one to execute :lwindow whenever :lgetexpr is executed.

:cwindow opens the quickfix window if there are valid entries in the quickfix list and :lwindow does the same for the location list and the location window.

We need both if we don't want to lose the flexibility of the original commands we are trying to replace.

augroup quickfix
    autocmd!
    autocmd QuickFixCmdPost cgetexpr cwindow
    autocmd QuickFixCmdPost lgetexpr lwindow
augroup END

Note that this snippet can be easily generalized to cover every quickfix command by changing the patterns cgetexpr and lgetexpr to [^l]* (to match commands that don't start with l) and l* (to match commands that start with l), respectively.

Enlightenment

So far, we have managed to:

• eliminate I/O-related usability issues,
• improve wildcard expansion,
• open the quickfix window automatically to show the results.

This leaves us us with a relatively straightforward snippet of vimcript and a much slicker experience:

function! Grep(...)
    return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
endfunction
command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)
augroup quickfix
    autocmd!
    autocmd QuickFixCmdPost cgetexpr cwindow
    autocmd QuickFixCmdPost lgetexpr lwindow
augroup END

That was, in a nutshell, where I was when this article, that took a different approach to the exact same problems, was posted on Reddit. The author's approach involved an abbreviation that allowed the user to type :grep as usual, and have it expanded to a more potent silent grep in an effort to reduce I/O-related noise.

While I don't use abbreviations much myself, I really like this idea of modifying the underlying behavior of common commands to unlock their potential. This is a strategy I have explored in the past that allows you to get more from the standard commands without having to learn new commands. You just do :foo bar, as usual, but you get a much more usable outcome.

It is that specific aspect that lacked in my lovely snippet: I had to use :Grep instead of :grep so it forced me to learn a new command (not a wildly different one, mind you, but still). Enlightenment came from Peter Rincker, who, in the spirit of the article, suggested the following missing piece:

cnoreabbrev <expr> grep (getcmdtype() ==# ':' && getcmdline() ==# 'grep') ? 'Grep' : 'grep'

which ties the whole thing up perfectly.

Now I can do the same :grep foo that I learned 10 years ago, but with a much cleaner and less labor-intensive user experience, and with a few lines of minimal, readable, maintainable configuration.

NOTE: While I consider this short snippet to be extremely useful, I don't pretend it to be a plug and play solution to all your problems, real or imaginary. I would prefer this experiment to inspire you to experiment with your own workflow than to be yanked and put in your vimrc. I can't stop you from doing that, of course, but oh well.

---

My Vim-related gists.

grep.vim

set grepprg=ag\ --vimgrep

function! Grep(...)
	return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
endfunction

command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)

cnoreabbrev <expr> grep  (getcmdtype() ==# ':' && getcmdline() ==# 'grep')  ? 'Grep'  : 'grep'
cnoreabbrev <expr> lgrep (getcmdtype() ==# ':' && getcmdline() ==# 'lgrep') ? 'LGrep' : 'lgrep'

augroup quickfix
	autocmd!
	autocmd QuickFixCmdPost cgetexpr cwindow
	autocmd QuickFixCmdPost lgetexpr lwindow
augroup END

@jssee I think I will update the snippet and the "article" to reflect this. Grepping the current file sounds like a common use case and the normal :grep foo % naturally expands the last % anyway, so, if the goal is to enhance :grep then this must be addressed.

I think this is 99% of the way there, but ideally quoted strings should be treated as a single argument so that it acts just like command line grep. That is, :Grep 'foo bar' file1 file2 file3 should treat the string foo bar as a single argument. As it stands right now, if I wanted to grep for foo % with :Grep 'foo %' file1 file2 file3 ... then the % would be expanded (since it's not part of the first argument as far as Vim is concerned).

Option flags (such as -w to search only for whole strings) should not be counted as arguments either. If I use :Grep -w PATTERN the -w should not be interpreted as the first argument, or else any %'s or #'s in the PATTERN will be expanded.

AFAIK Vim doesn't have a built-in way to split strings according to shell quotes. Some kind of "shell split" function is needed, i.e.:

shellsplit("'foo bar' baz qux") => ['foo bar', 'baz', 'qux']

Then the full function would be

function! Grep(args)
    let grep_cmd = [ &grepprg ]
    let full_args = shellsplit(a:args)
    let [opts, args] = s:separate_opts_from_args(full_args)
    let first_arg = [ args[0] ]
    let rest_args = join(args[1:-1], ' ')
    let expanded_rest_args = [ expandcmd(rest_args) ]
    let all_components = [grep_cmd] + [opts] + [first_arg] + [expanded_rest_args]
    let full_cmd = join(all_components, ' ')
    return system(full_cmd)
endfunction

I realize this is now getting fairly complicated. Is there any easier/simpler way to do this that I'm overlooking?

One possible implementation of shellsplit is

function! Shellsplit(str)
    return map(split(a:str, '\%(^\%("[^"]*"\|''[^'']*''\|[^"'']\)*\)\@<= '), {_, v -> substitute(v, '^["'']\|["'']$', '', 'g')})
endfunction

@gpanders the command passes the arguments as <f-args> which allows doing:

:Grep foo\ %

so it sounds to me like a non-issue.

@gpanders the command passes the arguments as <f-args> which allows doing:

:Grep foo\ %

so it sounds to me like a non-issue.

I understand it's not insurmountable, but inasmuch as it's important to you (which it may not be) to have :Grep usage in Vim mirror grep on the command line, backslash-escaping spaces is not an optimal solution.

I implemented the shellsplit and separate_opts_from_args functions I mentioned in my earlier post because I want :Grep to be just like grep, and I used what you wrote as a starting point, so thanks for doing this write up.

As I see it, :Grep should behave as closely as possible to how :grep behaves, with some leg room for eventual optimisations.

Wildcard expansion

With :grep (and other commands), wildcard expansion is done:

• before executing the command
• on everything that comes after the command name followed by a space.

Examples:

Before expansion	After expansion
:grep foo filename	:grep foo filename
:grep foo %	:grep foo filename
:grep foo \%	:grep foo %
:grep % %	:grep filename filename
:grep \% %	:grep % filename
:grep \% \%	:grep % %

Contrary to my initial idea I think I should keep that behaviour and don't try to come up with new idioms, at least regarding wildcard expansion. To that effect, passing all the arguments as-is to expandcmd() does the trick, and in a very simple way.

Quotes

With :grep, quotes are passed as-is to the external command so splitting the command into a first segment and a second one will obviously make this unnecessarily complicated.

Conclusion

So it looks to me like splitting is a bad idea and I should revert to a simpler and more standard-abiding pattern:

function! Grep(...)
	return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
endfunction

I noticed a typo.

diff --git a/grep.md b/grep.md
index d5c8ac8..72054c4 100644
--- a/grep.md
+++ b/grep.md
@@ -57,7 +57,7 @@ There are many ways to write the expression above, and choosing one or the other
 Expression | Performance | Stylish
 ---|---|---
 `&grepprg . ' ' . join(a:000, ' ')` | 0.000072s | No
-`join(extend([&grepprg, a:000), ' ')` | 0.000065s | Yes
+`join(extend([&grepprg], a:000), ' ')` | 0.000065s | Yes
 `join([&grepprg] + a:000, ' ')` | 0.000050s | Yes
 
 The third one it is, then:

Also out of curiosity is there a reason you explicitly join to use a separator of a single space when it is the default?

Thanks for spotting the typo. Also I tend to prefer explicit over implicit.

@romainl Thank you for the great gist. I am trying to use it with a word which contains a special character like $ but it doesn't return any results. How I can use it to search for $user for example? If I search from console line it works with:
$ ag '\$user'
but with Grep '\$user' there aren't any results.

@vaklinzi as is often the case with Vim's peculiar argument handling, I'm afraid you will have to use the dreaded triple-backslash hack, here:

:Grep '\\\$user'

which gets the job done in the most inelegant way possible.

--- EDIT ---

It looks like using :help fnameescape() before :help expandcmd() preserves the \ in $user so it may be a solution. It needs further testing, though.

Thank you for the fast response but it's still not working. Maybe I am doing something wrong...

@vaklinzi, I tested it before posting and it is definitely working, here:

To be 100% sure I tested it with $ vim -Nu grep.vim and it also worked, here:

FWIW:

$ ag --version
ag version 2.2.0

Features:
  +jit +lzma +zlib

Very strange. The only difference I see is that I use nvim instead vim and I changed the expandcmd() with expand() because it can't find the expandcmd() function.

I'm sorry but I only support Vim. Neovim and Vim are two different beasts and nothing that works in one should be assumed to work in the other.

@vaklinzi nvim-0.4.4 doesn't have expandcmd() but it is available if you use the most recent git build or the nightly build.

The difference is that expandcmd() will expand any wildcards in the provided string, whereas expand() will only do so if the wildcard is the first character. So you have to expand every argument passed to the function manually using map().

So we can do the following

" replace this
return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
" with this
return system(join([&grepprg] + [join(map(l:args, 'expand(v:val)'), ' ')], ' '))

However, this will expand some things we don't want expanded. For example, assume grepprg is rg --vimgrep and we have a directory containing main.py and lib.py.

If we run, :Grep print *.py, we expect the command run to be rg --vimgrep print *.py, which is what happens when using the original version. However, using the modified version, we get rg --vimgrep print lib.py \nmain.py.

The newline is not a mistake but it is what happens on my system and throws an error as the underlying shell receives two commands. Still not sure why this happens. However, either way, expand() expands the * before passing it to system(), but we don't want that.

So we can use a Funcref to filter out any args to :Grep that have a leading *. I'm sure there are other potential issues from using this method, but ideally those issues can be filtered out in CustomExpand().

Full replacement solution is below. I also added in a line to fallback to the original version if on nvim-0.5 or regular vim.

function! CustomExpand(val)
    " if starts with *, don't expand it
    if a:val =~ '^\*'
        return a:val
    else
        return expand(a:val)
    endif
endfunction

" call grepprg in a system shell instead of internal shell
function! ImprovedGrep(...)
    " expandcmd() is only supported in regular vim or nvim-0.5
    if has('nvim-0.5') || !has('nvim')
        return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
    else
        let l:args = copy(a:000)
        let CExp = function("CustomExpand")
        return system(join([&grepprg] + [join(map(l:args, 'CExp(v:val)'), ' ')], ' '))
    endif
endfunction

This is great! I modified it a bit to get a nicer quickfix/location window title (e.g. :rg --vimgrep whatever instead of :Grep("whatever")):

function! Grep(...)
    let s:command = join([&grepprg] + [expandcmd(join(a:000, ' '))], ' ')
    return system(s:command)
endfunction

...

augroup quickfix
    autocmd!
    autocmd QuickFixCmdPost cgetexpr cwindow
                \| call setqflist([], 'a', {'title': ':' . s:command})
    autocmd QuickFixCmdPost lgetexpr lwindow
                \| call setloclist(0, [], 'a', {'title': ':' . s:command})
augroup END

I just had an echo s:command before the return statement but this is a lot cleaner. Thanks!

Regarding the errorformat setting. https://github.com/fatih/vim-go automatically sets it in .go files, which is a bit intrusive, so I came up with this approach of temorarily setting it to grepformat for grep commands.

I also don't care much about inputting default command names, because I prefer to map such common operations.

function! s:with_grep_format(cmd, prg, args, path) abort
    let l:saved_errorformat = &errorformat
    let &errorformat = &grepformat
    try
        if a:path ==# ''
            exe a:cmd . ' system("' . a:prg . ' ' . a:args . ' ")'
        else
            exe a:cmd . ' system("' . a:prg . ' ' . a:args . ' ' . shellescape(a:path) . '")'
        endif
    finally
        let &errorformat = l:saved_errorformat
    endtry
endfunction

command! -complete=file_in_path -nargs=+ GrepProject call s:with_grep_format('cgetexpr', &grepprg, expandcmd(<q-args>), '')

command! -nargs=+ LGrepDir call s:with_grep_format('lgetexpr', &grepprg, <q-args>, expand('%:p:h'))
command! -complete=file_in_path -nargs=+ LGrepProject call s:with_grep_format('lgetexpr', &grepprg, expandcmd(<q-args>), '')
command! -nargs=+ LGrepFile call s:with_grep_format('lgetexpr', &grepprg, <q-args>, expand('%:p'))

nnoremap <leader>sp :<c-u>GrepProject ''<left>
nnoremap <leader>lp :<c-u>LGrepProject ''<left>
nnoremap <leader>ld :<c-u>LGrepDir ''<left>
nnoremap <leader>lf :<c-u>LGrepFile ''<left>

@EgZvor switching to &grepformat is a cool idea.

Sorry to toot my own horn but I'm rather proud of the plugin I made for silent :grep and :make replacements in Vim. It will use &grepformat or &errorformat accordingly.

https://github.com/george-b/sf.vim

Isn't it always better to define s: functions and call them using <SID>?

@lxvs it's not.

It certainly is a good practice to do so when writing a distributable plugin and if the design of the plugin dictates it, but not always. In my own plugins, I tend to favor <Plug> mappings and autoloaded functions to that whole s:/<SID> pattern.

I use this approach for quite some time now and am rather satisfied with it. If not just for the reason that I can avoid fuzzy finders that I do not particularly like anyway. I have one issue though. For some reason Vim can't seem to detect the filetype of the first item of the Quickfixlist, which is rather annoying. Do you have an idea what could be the reason for this rather strange behavior? I tested this in multiple directories so it seems to be independent off the specific filetype.

@mxngls I didn't notice such a problem. Could you whip up a minimal setup? Also, what Vim version do you use?

@romainl I played around a bit and I think the reason is that I used copen instead of cwindow in a custom function that I'd call when opening a Quickfix list. When using cwindow the problem described above does not occur. Using your setup as showcased above and just changing cwindow with copen should suffice to reproduce.
Reading the documentation for both copen and cwindow I don't see any logical reason for this behavior and thus conclude that it is a bug.

I tested this on Vim 9.0 as well as Neovim 0.9.0.

---

Edit Sun Oct 1 11:54 AM

Using cindow or cindow leads to the same behavior as described in my first comment above.