Skip to content

Instantly share code, notes, and snippets.

@romainl
Last active November 3, 2024 17:56
Show Gist options
  • Save romainl/56f0c28ef953ffc157f36cc495947ab3 to your computer and use it in GitHub Desktop.
Save romainl/56f0c28ef953ffc157f36cc495947ab3 to your computer and use it in GitHub Desktop.
Instant grep + quickfix

FOREWORDS

I don't mean the snippet at the bottom of this gist to be a generic plug-n-play solution to your search needs. It is very likely to not work for you or even break things, and it certainly is not as extensively tested and genericised as your regular third-party plugin.

My goal, here and in most of my posts, is to show how Vim's features can be leveraged to build your own high-level, low-maintenance, workflows without systematically jumping on the plugins bandwagon or twisting Vim's arm.


Instant grep + quickfix

:help :grep is a fantastic tool, especially when combined with the quickfix, but it is certainly a bit rough around the edges. Let's see what it takes to smooth it out.

Tell Vim what external command to use for grepping

By default, Vim is set to use grep for :grep, which seems logical given the name of the command and all. But it also lets us customize the actual command via :help 'grepprg'. We can use it to add default parameters, for example, or to point Vim to a completely different program.

Over the years, people have been busy working on faster and/or more developer-friendly grep alternatives. The first such effort that I remember of was ack, then ag changed the rules of the game and other players entered the field, the latest being rg. Depending on your use cases, switching :grep to one of those alternatives may be a wise choice.

In the example below I use ag but feel free to substitute it with whatever you want.

set grepprg=ag\ --vimgrep

Now, this change is not strictly necessary for what we are trying to do but, even if we stopped here, using ag or rg over grep can be a cheap and non-negligible upgrade to the built-in :grep and :lgrep.

Perform the search in a sub-shell

One of those rough edges I alluded to earlier is the fact that Vim executes the command in Vim's parent shell. This has a few consequences that contribute to a less than ideal experience:

  • Vim is suspended for how long it takes the command to finish.
  • The results are printed in the shell and clog our terminal's scroll buffer.
  • We must press the dreaded <CR> to come back to Vim.

Enters a sister command of :grep, :help :cgetexpr, that takes an "expression" as input. In concrete terms, :cgetexpr takes the raw output of grep or ag or whatever, parses it according to :help 'errorformat' and creates a quickfix list. :cgetexpr operates at a lower level than :grep and doesn't involve any I/O so it is simpler and faster.

:cgetexpr <expression>

Now, how do we get the output of ag if we can't execute it in the shell? Via :help system() of course:

:cgetexpr system('ag foo')

But that's a lot to type, right? Well, how about wrapping it in a custom function?

function! Grep(args)
    return system('ag ' . a:args)
endfunction

Now we have the following, which seems to solve all of the issues outlined above:

:cgetexpr Grep('foo')

But we already have ag set as our 'grepprg' so we might as well use that, making our function future-proof:

function! Grep(args)
    return system(&grepprg . ' ' . a:args)
endfunction

From there, we can modify the function to handle multiple arguments because :grep 'foo bar' filename is a very reasonable thing to do:

function! Grep(...)
    return system(&grepprg . ' ' . join(a:000, ' '))
endfunction

There are many ways to write the expression above, and choosing one or the other is mostly a matter of personal preference… and performance:

Expression Performance Stylish
&grepprg . ' ' . join(a:000, ' ') 0.000072s No
join(extend([&grepprg], a:000), ' ') 0.000065s Yes
join([&grepprg] + a:000, ' ') 0.000050s Yes

The third one it is, then:

function! Grep(...)
    return system(join([&grepprg] + a:000, ' '))
endfunction

What this does is:

  • concatenate a list with &grepprg as sole value with a list containing all the arguments passed to the function,
  • join the resulting list into a single string,
  • pass it to system().

Nice… Our little function works very well but calling it manually kind of defeats the point we are trying to make.

Create a command to populate the quickfix list

We will start small and simple with a simple :Grep command that takes one or more arguments:

command! -nargs=+ Grep cgetexpr grep(<f-args>)

which seems to do the job:

:Grep foo

but we might want to restrict our search to a given directory or file so it might be a good idea to use command-line completion:

command! -nargs=+ -complete=file_in_path Grep cgetexpr Grep(<f-args>)

and we might want to execute another command after :Grep foo, so we should add another flag:

command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)

and let's not forget :help :lgrep:

command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)

Aaaaand we are done with the usability issues mentioned above. Here is our little snippet in all its glory:

function! Grep(...)
    return system(join([&grepprg] + a:000), ' '))
endfunction
command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)

Dealing with not-so-edge case

Our custom :Grep command seems to do a good job: :Grep foo *.js is faster than ever and we don't have to press <Enter> anymore. This is all good but :grep expands wildcards by default (think :grep foo % or :grep bar ##) and we don't want to lose that, so we will have to modify our expression slightly:

return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))

Here we use :help expandcmd() to expand any glob present in the "arguments" part of the command before it is executed, which makes :Grep foo % work like :grep foo %.

Neat!

Open the location/quickfix window automatically if there are valid entries in the list

There is one usability issue that we haven't mentioned before: when the search is done, we get a little message about the first item in our search but that's all. True, we can use :help :cn and friends to jump around quickfix entries but, if we want to see the results, we have to use :help :cwindow:

:Grep foo
:cwindow

or, shorter:

:Gr foo | cw

What if we could open the quickfix window automatically when the quickfix list changes?

Autocommands to the rescue! The first step is to define a self-clearing group so that our autocommands don't needlessly pile up if we re-:source our vimrc:

augroup quickfix
    autocmd!
    " our autocommands here
augroup END

Then, we add two autocommands:

  • one to execute :cwindow whenever :cgetexpr is executed,
  • another one to execute :lwindow whenever :lgetexpr is executed.

:cwindow opens the quickfix window if there are valid entries in the quickfix list and :lwindow does the same for the location list and the location window.

We need both if we don't want to lose the flexibility of the original commands we are trying to replace.

augroup quickfix
    autocmd!
    autocmd QuickFixCmdPost cgetexpr cwindow
    autocmd QuickFixCmdPost lgetexpr lwindow
augroup END

Note that this snippet can be easily generalized to cover every quickfix command by changing the patterns cgetexpr and lgetexpr to [^l]* (to match commands that don't start with l) and l* (to match commands that start with l), respectively.

Enlightenment

So far, we have managed to:

  • eliminate I/O-related usability issues,
  • improve wildcard expansion,
  • open the quickfix window automatically to show the results.

This leaves us us with a relatively straightforward snippet of vimscript and a much slicker experience:

function! Grep(...)
    return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
endfunction
command! -nargs=+ -complete=file_in_path -bar Grep  cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)
augroup quickfix
    autocmd!
    autocmd QuickFixCmdPost cgetexpr cwindow
    autocmd QuickFixCmdPost lgetexpr lwindow
augroup END

That was, in a nutshell, where I was when this article, that took a different approach to the exact same problems, was posted on Reddit. The author's approach involved an abbreviation that allowed the user to type :grep as usual, and have it expanded to a more potent silent grep in an effort to reduce I/O-related noise.

While I don't use abbreviations much myself, I really like this idea of modifying the underlying behavior of common commands to unlock their potential. This is a strategy I have explored in the past that allows you to get more from the standard commands without having to learn new commands. You just do :foo bar, as usual, but you get a much more usable outcome.

It is that specific aspect that lacked in my lovely snippet: I had to use :Grep instead of :grep so it forced me to learn a new command (not a wildly different one, mind you, but still). Enlightenment came from Peter Rincker, who, in the spirit of the article, suggested the following missing piece:

cnoreabbrev <expr> grep (getcmdtype() ==# ':' && getcmdline() ==# 'grep') ? 'Grep' : 'grep'

which ties the whole thing up perfectly.

Now I can do the same :grep foo that I learned 10 years ago, but with a much cleaner and less labor-intensive user experience, and with a few lines of minimal, readable, maintainable configuration.

grep

NOTE: While I consider this short snippet to be extremely useful, I don't pretend it to be a plug and play solution to all your problems, real or imaginary. I would prefer this experiment to inspire you to experiment with your own workflow than to be yanked and put in your vimrc. I can't stop you from doing that, of course, but oh well.


My Vim-related gists.

set grepprg=ag\ --vimgrep
function! Grep(...)
return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
endfunction
command! -nargs=+ -complete=file_in_path -bar Grep cgetexpr Grep(<f-args>)
command! -nargs=+ -complete=file_in_path -bar LGrep lgetexpr Grep(<f-args>)
cnoreabbrev <expr> grep (getcmdtype() ==# ':' && getcmdline() ==# 'grep') ? 'Grep' : 'grep'
cnoreabbrev <expr> lgrep (getcmdtype() ==# ':' && getcmdline() ==# 'lgrep') ? 'LGrep' : 'lgrep'
augroup quickfix
autocmd!
autocmd QuickFixCmdPost cgetexpr cwindow
autocmd QuickFixCmdPost lgetexpr lwindow
augroup END
@romainl
Copy link
Author

romainl commented Apr 23, 2020

As I see it, :Grep should behave as closely as possible to how :grep behaves, with some leg room for eventual optimisations.

Wildcard expansion

With :grep (and other commands), wildcard expansion is done:

  • before executing the command
  • on everything that comes after the command name followed by a space.

Examples:

Before expansion After expansion
:grep foo filename :grep foo filename
:grep foo % :grep foo filename
:grep foo \% :grep foo %
:grep % % :grep filename filename
:grep \% % :grep % filename
:grep \% \% :grep % %

Contrary to my initial idea I think I should keep that behaviour and don't try to come up with new idioms, at least regarding wildcard expansion. To that effect, passing all the arguments as-is to expandcmd() does the trick, and in a very simple way.

Quotes

With :grep, quotes are passed as-is to the external command so splitting the command into a first segment and a second one will obviously make this unnecessarily complicated.

Conclusion

So it looks to me like splitting is a bad idea and I should revert to a simpler and more standard-abiding pattern:

function! Grep(...)
	return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
endfunction

@g0xA52A2A
Copy link

I noticed a typo.

diff --git a/grep.md b/grep.md
index d5c8ac8..72054c4 100644
--- a/grep.md
+++ b/grep.md
@@ -57,7 +57,7 @@ There are many ways to write the expression above, and choosing one or the other
 Expression | Performance | Stylish
 ---|---|---
 `&grepprg . ' ' . join(a:000, ' ')` | 0.000072s | No
-`join(extend([&grepprg, a:000), ' ')` | 0.000065s | Yes
+`join(extend([&grepprg], a:000), ' ')` | 0.000065s | Yes
 `join([&grepprg] + a:000, ' ')` | 0.000050s | Yes
 
 The third one it is, then:

Also out of curiosity is there a reason you explicitly join to use a separator of a single space when it is the default?

@romainl
Copy link
Author

romainl commented Jun 7, 2020

Thanks for spotting the typo. Also I tend to prefer explicit over implicit.

@vaklinzi
Copy link

vaklinzi commented Jul 9, 2020

@romainl Thank you for the great gist. I am trying to use it with a word which contains a special character like $ but it doesn't return any results. How I can use it to search for $user for example? If I search from console line it works with:
$ ag '\$user'
but with Grep '\$user' there aren't any results.

@romainl
Copy link
Author

romainl commented Jul 9, 2020

@vaklinzi as is often the case with Vim's peculiar argument handling, I'm afraid you will have to use the dreaded triple-backslash hack, here:

:Grep '\\\$user'

which gets the job done in the most inelegant way possible.

--- EDIT ---

It looks like using :help fnameescape() before :help expandcmd() preserves the \ in $user so it may be a solution. It needs further testing, though.

@vaklinzi
Copy link

vaklinzi commented Jul 9, 2020

Kapture 2020-07-09 at 16 02 42

Thank you for the fast response but it's still not working. Maybe I am doing something wrong...

@romainl
Copy link
Author

romainl commented Jul 9, 2020

@vaklinzi, I tested it before posting and it is definitely working, here: :Grep '\$user'

To be 100% sure I tested it with $ vim -Nu grep.vim and it also worked, here: :Grep '\$user'

FWIW:

$ ag --version
ag version 2.2.0

Features:
  +jit +lzma +zlib

@vaklinzi
Copy link

vaklinzi commented Jul 9, 2020

Very strange. The only difference I see is that I use nvim instead vim and I changed the expandcmd() with expand() because it can't find the expandcmd() function.

image

@romainl
Copy link
Author

romainl commented Jul 9, 2020

I'm sorry but I only support Vim. Neovim and Vim are two different beasts and nothing that works in one should be assumed to work in the other.

@maujim
Copy link

maujim commented Apr 12, 2021

@vaklinzi nvim-0.4.4 doesn't have expandcmd() but it is available if you use the most recent git build or the nightly build.

The difference is that expandcmd() will expand any wildcards in the provided string, whereas expand() will only do so if the wildcard is the first character. So you have to expand every argument passed to the function manually using map().

So we can do the following

" replace this
return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
" with this
return system(join([&grepprg] + [join(map(l:args, 'expand(v:val)'), ' ')], ' '))

However, this will expand some things we don't want expanded. For example, assume grepprg is rg --vimgrep and we have a directory containing main.py and lib.py.

If we run, :Grep print *.py, we expect the command run to be rg --vimgrep print *.py, which is what happens when using the original version. However, using the modified version, we get rg --vimgrep print lib.py \nmain.py.

The newline is not a mistake but it is what happens on my system and throws an error as the underlying shell receives two commands. Still not sure why this happens. However, either way, expand() expands the * before passing it to system(), but we don't want that.

So we can use a Funcref to filter out any args to :Grep that have a leading *. I'm sure there are other potential issues from using this method, but ideally those issues can be filtered out in CustomExpand().

Full replacement solution is below. I also added in a line to fallback to the original version if on nvim-0.5 or regular vim.

function! CustomExpand(val)
    " if starts with *, don't expand it
    if a:val =~ '^\*'
        return a:val
    else
        return expand(a:val)
    endif
endfunction

" call grepprg in a system shell instead of internal shell
function! ImprovedGrep(...)
    " expandcmd() is only supported in regular vim or nvim-0.5
    if has('nvim-0.5') || !has('nvim')
        return system(join([&grepprg] + [expandcmd(join(a:000, ' '))], ' '))
    else
        let l:args = copy(a:000)
        let CExp = function("CustomExpand")
        return system(join([&grepprg] + [join(map(l:args, 'CExp(v:val)'), ' ')], ' '))
    endif
endfunction

@jasonccox
Copy link

jasonccox commented Aug 19, 2021

This is great! I modified it a bit to get a nicer quickfix/location window title (e.g. :rg --vimgrep whatever instead of :Grep("whatever")):

function! Grep(...)
    let s:command = join([&grepprg] + [expandcmd(join(a:000, ' '))], ' ')
    return system(s:command)
endfunction

...

augroup quickfix
    autocmd!
    autocmd QuickFixCmdPost cgetexpr cwindow
                \| call setqflist([], 'a', {'title': ':' . s:command})
    autocmd QuickFixCmdPost lgetexpr lwindow
                \| call setloclist(0, [], 'a', {'title': ':' . s:command})
augroup END

@maujim
Copy link

maujim commented Aug 19, 2021

I just had an echo s:command before the return statement but this is a lot cleaner. Thanks!

@EgZvor
Copy link

EgZvor commented May 18, 2022

Regarding the errorformat setting. https://github.com/fatih/vim-go automatically sets it in .go files, which is a bit intrusive, so I came up with this approach of temorarily setting it to grepformat for grep commands.

I also don't care much about inputting default command names, because I prefer to map such common operations.

function! s:with_grep_format(cmd, prg, args, path) abort
    let l:saved_errorformat = &errorformat
    let &errorformat = &grepformat
    try
        if a:path ==# ''
            exe a:cmd . ' system("' . a:prg . ' ' . a:args . ' ")'
        else
            exe a:cmd . ' system("' . a:prg . ' ' . a:args . ' ' . shellescape(a:path) . '")'
        endif
    finally
        let &errorformat = l:saved_errorformat
    endtry
endfunction

command! -complete=file_in_path -nargs=+ GrepProject call s:with_grep_format('cgetexpr', &grepprg, expandcmd(<q-args>), '')

command! -nargs=+ LGrepDir call s:with_grep_format('lgetexpr', &grepprg, <q-args>, expand('%:p:h'))
command! -complete=file_in_path -nargs=+ LGrepProject call s:with_grep_format('lgetexpr', &grepprg, expandcmd(<q-args>), '')
command! -nargs=+ LGrepFile call s:with_grep_format('lgetexpr', &grepprg, <q-args>, expand('%:p'))

nnoremap <leader>sp :<c-u>GrepProject ''<left>
nnoremap <leader>lp :<c-u>LGrepProject ''<left>
nnoremap <leader>ld :<c-u>LGrepDir ''<left>
nnoremap <leader>lf :<c-u>LGrepFile ''<left>

@romainl
Copy link
Author

romainl commented May 18, 2022

@EgZvor switching to &grepformat is a cool idea.

@g0xA52A2A
Copy link

Sorry to toot my own horn but I'm rather proud of the plugin I made for silent :grep and :make replacements in Vim. It will use &grepformat or &errorformat accordingly.

https://github.com/george-b/sf.vim

@lxvs
Copy link

lxvs commented Jul 15, 2022

Isn't it always better to define s: functions and call them using <SID>?

@romainl
Copy link
Author

romainl commented Jul 15, 2022

@lxvs it's not.

It certainly is a good practice to do so when writing a distributable plugin and if the design of the plugin dictates it, but not always. In my own plugins, I tend to favor <Plug> mappings and autoloaded functions to that whole s:/<SID> pattern.

@mxngls
Copy link

mxngls commented Sep 29, 2023

I use this approach for quite some time now and am rather satisfied with it. If not just for the reason that I can avoid fuzzy finders that I do not particularly like anyway. I have one issue though. For some reason Vim can't seem to detect the filetype of the first item of the Quickfixlist, which is rather annoying. Do you have an idea what could be the reason for this rather strange behavior? I tested this in multiple directories so it seems to be independent off the specific filetype.

@romainl
Copy link
Author

romainl commented Sep 29, 2023

@mxngls I didn't notice such a problem. Could you whip up a minimal setup? Also, what Vim version do you use?

@mxngls
Copy link

mxngls commented Oct 1, 2023

@romainl I played around a bit and I think the reason is that I used copen instead of cwindow in a custom function that I'd call when opening a Quickfix list. When using cwindow the problem described above does not occur. Using your setup as showcased above and just changing cwindow with copen should suffice to reproduce.
Reading the documentation for both copen and cwindow I don't see any logical reason for this behavior and thus conclude that it is a bug.

I tested this on Vim 9.0 as well as Neovim 0.9.0.


Edit Sun Oct 1 11:54 AM

Using cindow or cindow leads to the same behavior as described in my first comment above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment