Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Speed up zsh compinit by only checking cache once a day.
# On slow systems, checking the cached .zcompdump file to see if it must be
# regenerated adds a noticable delay to zsh startup. This little hack restricts
# it to once a day. It should be pasted into your own completion file.
#
# The globbing is a little complicated here:
# - '#q' is an explicit glob qualifier that makes globbing work within zsh's [[ ]] construct.
# - 'N' makes the glob pattern evaluate to nothing when it doesn't match (rather than throw a globbing error)
# - '.' matches "regular files"
# - 'mh+24' matches files (or directories or whatever) that are older than 24 hours.
autoload -Uz compinit
if [[ -n ${ZDOTDIR}/.zcompdump(#qN.mh+24) ]]; then
compinit;
else
compinit -C;
fi;
@gartrog
Copy link

gartrog commented Feb 13, 2019

@aksh1618 I believe it should be -mtime +1 to match files older than a day

@igorepst
Copy link

igorepst commented Apr 20, 2019

My version. Thanks to @emilyst and this snippet from prezto

_zpcompinit_custom() {
  setopt extendedglob local_options
  autoload -Uz compinit
  local zcd=${ZDOTDIR:-$HOME}/.zcompdump
  local zcdc="$zcd.zwc"
  # Compile the completion dump to increase startup speed, if dump is newer or doesn't exist,
  # in the background as this is doesn't affect the current session
  if [[ -f "$zcd"(#qN.m+1) ]]; then
        compinit -i -d "$zcd"
        { rm -f "$zcdc" && zcompile "$zcd" } &!
  else
        compinit -C -d "$zcd"
        { [[ ! -f "$zcdc" || "$zcd" -nt "$zcdc" ]] && rm -f "$zcdc" && zcompile "$zcd" } &!
  fi
}

Features:

  1. Implicit plus sign in glob didn't work for me, so I added it explicitly
  2. extendedglob is local inside the function
  3. zcompile is run in the background
  4. Somehow the compiled file changed his size pretty much (my config problem?), so I preferred to delete it explicitly

Usage

  1. Just call _zpcompinit_custom
  2. Or, if you use zplugin
  • Change zcd to local zcd=${ZPLGM[ZCOMPDUMP_PATH]:-${ZDOTDIR:-$HOME}/.zcompdump}
  • And call like this, according to the used plugin:
zplugin ice depth'1' lucid wait'0' atinit"_zpcompinit_custom; zpcdreplay"
zplugin light zdharma/fast-syntax-highlighting

@goldylucks
Copy link

goldylucks commented Jul 24, 2019

It should be pasted into your own completion file.

@ctechhols is there a way to load it in once place?

I have dozens of those:

echo $fpath
/usr/lib/heroku/node_modules/@heroku-cli/plugin-autocomplete/autocomplete/zsh /home/linuxbrew/.linuxbrew/share/zsh/site-functions /home/goldy/.oh-my-zsh/plugins/heroku /home/goldy/.oh-my-zsh/plugins/colorize /home/goldy/.oh-my-zsh/plugins/colored-man-pages /home/goldy/.oh-my-zsh/plugins/git-extras /home/goldy/.oh-my-zsh/plugins/autojump /home/goldy/.oh-my-zsh/plugins/git /home/goldy/Dropbox/Mackup/custom-oh-my-zsh/plugins/yarn-autocompletions /home/goldy/.oh-my-zsh/functions /home/goldy/.oh-my-zsh/completions /usr/local/share/zsh/site-functions /usr/share/zsh/vendor-functions /usr/share/zsh/vendor-completions /usr/share/zsh/functions/Calendar /usr/share/zsh/functions/Chpwd /usr/share/zsh/functions/Completion /usr/share/zsh/functions/Completion/AIX /usr/share/zsh/functions/Completion/BSD /usr/share/zsh/functions/Completion/Base /usr/share/zsh/functions/Completion/Cygwin /usr/share/zsh/functions/Completion/Darwin /usr/share/zsh/functions/Completion/Debian /usr/share/zsh/functions/Completion/Linux /usr/share/zsh/functions/Completion/Mandriva /usr/share/zsh/functions/Completion/Redhat /usr/share/zsh/functions/Completion/Solaris /usr/share/zsh/functions/Completion/Unix /usr/share/zsh/functions/Completion/X /usr/share/zsh/functions/Completion/Zsh /usr/share/zsh/functions/Completion/openSUSE /usr/share/zsh/functions/Exceptions /usr/share/zsh/functions/MIME /usr/share/zsh/functions/Math /usr/share/zsh/functions/Misc /usr/share/zsh/functions/Newuser /usr/share/zsh/functions/Prompts /usr/share/zsh/functions/TCP /usr/share/zsh/functions/VCS_Info /usr/share/zsh/functions/VCS_Info/Backends /usr/share/zsh/functions/Zftp /usr/share/zsh/functions/Zle

@ctechols
Copy link
Author

ctechols commented Jul 24, 2019

@goldylucks

It should be pasted into your own completion file.

This statement just means whichever zsh config file initializes your completion system. It will be the file that currently contains the compinit invocation. This would typically be ~/.zshrc, but may vary depending on how you ran compinstall.

I actually don't use or advocate this snippet anymore. Compinit's cache rebuilding mechanism works by counting the number of completion definitions and rebuilding the cache if that number has changed. This snippet was useful in situations where filesystem access was slow -- such as cygwin with an HDD. If you are using Mac, linux or WSL on an SSD I doubt you will notice any difference by enabling this.

@dropwhile
Copy link

dropwhile commented Aug 20, 2019

It did help for my startup times (by quite a bit). That is, once I figured out that compinit does not modify the zcompdump file if completions haven't changed, so I ended up having a slow path until I added a separate file that I use to determine if compinit was called and just didn't update the file. A separate file so I didn't also need to force zcompilation again if I didn't need to.

## completion stuff
zstyle ':compinstall' filename '$HOME/.zshrc'

zcachedir="$HOME/.zcache"
[[ -d "$zcachedir" ]] || mkdir -p "$zcachedir"

_update_zcomp() {
    setopt local_options
    setopt extendedglob
    autoload -Uz compinit
    local zcompf="$1/zcompdump"
    # use a separate file to determine when to regenerate, as compinit doesn't
    # always need to modify the compdump
    local zcompf_a="${zcompf}.augur"

    if [[ -e "$zcompf_a" && -f "$zcompf_a"(#qN.md-1) ]]; then
        compinit -C -d "$zcompf"
    else
        compinit -d "$zcompf"
        touch "$zcompf_a"
    fi
    # if zcompdump exists (and is non-zero), and is older than the .zwc file,
    # then regenerate
    if [[ -s "$zcompf" && (! -s "${zcompf}.zwc" || "$zcompf" -nt "${zcompf}.zwc") ]]; then
        # since file is mapped, it might be mapped right now (current shells), so
        # rename it then make a new one
        [[ -e "$zcompf.zwc" ]] && mv -f "$zcompf.zwc" "$zcompf.zwc.old"
        # compile it mapped, so multiple shells can share it (total mem reduction)
        # run in background
        zcompile -M "$zcompf" &!
    fi
}
_update_zcomp "$zcachedir"
unfunction _update_zcomp

@lervag
Copy link

lervag commented Sep 26, 2019

How about simply using

autoload -Uz compinit
if [[ -n ~/.zcompdump(#qN.mh+24) ]]; then
  compinit
  touch .zcompdump
else
  compinit -C
fi

I.e. add touch .zcompdump? It seems to work well for me, but am I missing anything?

@dropwhile
Copy link

dropwhile commented Sep 26, 2019

@lervag that works fine, but in my case i'm also zcompiling the compdump as a separate step, and I didn't want to do the zcompile step if it isn't necessary (if the dump hadn't changed). If you aren't doing that, then I think your much simpler version is great.

@lervag
Copy link

lervag commented Sep 26, 2019

@cactus Thanks for the quick reply and reassurance. I see I will be missing out in the sense that I also notice that most often, the compinit call changes nothing and takes a long time. Still, once a day doesn't matter that much to me. So I'm happy with the simple version.

Again, thanks!

@rongmu
Copy link

rongmu commented Dec 12, 2019

Another solution if extendedglob is not wanted, using anonymous function:

autoload -Uz compinit

() {
  if [[ $# -gt 0 ]]; then
    compinit
  else
    compinit -C
  fi
} ${ZDOTDIR:-$HOME}/.zcompdump(N.mh+24)

Or, enable extendedglob locally to the function:

() {
  setopt extendedglob local_options

  if [[ -n ${ZDOTDIR:-$HOME}/.zcompdump(#qN.mh+24) ]]; then
    compinit
  else
    compinit -C
  fi
}

@dm17
Copy link

dm17 commented Dec 16, 2019

Can it be made to outperform dash and mksh latency wise?

@seanbreckenridge
Copy link

seanbreckenridge commented Feb 5, 2020

appreciate all the discussion in this thread; tried most of the solutions here and they end up slowing down my startup time by a decent amount.

I would think this has something to do with my zcompdump being relatively small (~2000 lines)

@ondt
Copy link

ondt commented May 12, 2020

Is there something wrong with running compinit in the background?

autoload -U compinit && (compinit &)

@lervag
Copy link

lervag commented May 12, 2020

Is there something wrong with running compinit in the background?

Hah, I never thought of that! Very good question, I would love to hear an answer from someone who knows the details of compinit and its implementation!

@ondt
Copy link

ondt commented May 12, 2020

Nevermind, it doesn't work 😸
I found out that $ZSH/oh-my-zsh.sh already runs compinit, so by adding another compinit I just slowed it down even more.

@emilyst
Copy link

emilyst commented May 13, 2020

Is there something wrong with running compinit in the background?

autoload -U compinit && (compinit &)

It's an interesting idea! Unfortunately, compinit is designed to initialize the current session's completions. See the manual for more about that.

@matthewbauer
Copy link

matthewbauer commented Jul 6, 2020

autoload -U compinit && (compinit &; compinit -C)

might work

@sparcbr
Copy link

sparcbr commented Aug 3, 2020

A few notes about the this gist..
The file age test code seems to work even without #q or setopt extendedglob. Not quite sure why, but would need to check the zsh man page.

# 3 hours and older .zcompdump:
❯ [[ -n ${ZDOTDIR:-$HOME}/.zcompdump(N.mh+2) ]] &&  echo yes       
yes

Skipping Ubuntu system-wide compinit

Don't forget to check if the system wide zshrc is running compinit, and disable it if you can. So you can run your own compinit startup code.
Ubuntu does this in /etc/zsh/zshrc, and you can disable by adding this to your ~/.zshenv:

# Skip the not really helping Ubuntu global compinit
skip_global_compinit=1

Using Zinit - a fast Zsh plugin manager

Zinit provides a way to intercept compdef calls (used completion by completion scripts) to execute them later, after compinit, using the zinit cdreplay command:

# ~/.zshrc
source ~/.zinit/bin/zinit.zsh
zinit load "some/plugin"
zinit load "other/plugin"
autoload -Uz compinit; compinit
zinit cdreplay

More info:
https://github.com/zdharma/zinit#calling-compinit-without-turbo-mode

@faaizajaz
Copy link

faaizajaz commented Sep 8, 2020

I am totally confused. Where is everyone adding these snippets? I have tried adding to my .zshrc to no success. The original gist mentions "zsh completion file" but what is that? Google turns up nothing.

@duraki
Copy link

duraki commented Oct 9, 2020

I agree w/ @faaizajaz question; where do we put this stuff?

@ctechols
Copy link
Author

ctechols commented Oct 9, 2020

@faaizajaz, @duraki I can't speak with specifics about most of the additional discussion on this gist. (I no longer use any of these approaches) In general though, you are going to want to make your changes right after you load the compinit module. This is likely to be the file that compinstall created or edited.

To find this file on your system try zstyle | grep compinstall

@duraki
Copy link

duraki commented Oct 11, 2020

Thanks @ctechols

This is how I did it.

I created this folder:

  • mkdir ~/.oh-my-zsh/plugins/zshfl

Inside I've put:

  • touch zshfl.plugin.zsh
# compinit optimization for oh-my-zsh
# On slow systems, checking the cached .zcompdump file to see if it must be
# regenerated adds a noticable delay to zsh startup.  This little hack restricts
# it to once a day.  It should be pasted into your own completion file.
#
# The globbing is a little complicated here:
# - '#q' is an explicit glob qualifier that makes globbing work within zsh's [[ ]] construct.
# - 'N' makes the glob pattern evaluate to nothing when it doesn't match (rather than throw a globbing error)
# - '.' matches "regular files"
# - 'mh+24' matches files (or directories or whatever) that are older than 24 hours.
#autoload -U compaudit compinit
#: ${ZSH_DISABLE_COMPFIX:=true}
# ...
setopt extendedglob
if [[ $ZSH_DISABLE_COMPFIX != true ]]; then
# If completion insecurities exist, warn the user without enabling completions.
if ! compaudit &>/dev/null; then
# This function resides in the "lib/compfix.zsh" script sourced above.
    handle_completion_insecurities
# Else, enable and cache completions to the desired file.
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
      compinit -d "${ZSH_COMPDUMP}"
      compdump
else
      compinit -C
fi
fi
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
    compinit -i -d "${ZSH_COMPDUMP}"
    compdump
else
    compinit -C
fi
fi

And I've loaded it through .zshrc in plugins=(zshfs git ...)

@ftlno
Copy link

ftlno commented Oct 27, 2020

Thanks @ctechols

This is how I did it.

I created this folder:

  • mkdir ~/.oh-my-zsh/plugins/zshfl

Inside I've put:

  • touch zshfl.plugin.zsh
# compinit optimization for oh-my-zsh
# On slow systems, checking the cached .zcompdump file to see if it must be
# regenerated adds a noticable delay to zsh startup.  This little hack restricts
# it to once a day.  It should be pasted into your own completion file.
#
# The globbing is a little complicated here:
# - '#q' is an explicit glob qualifier that makes globbing work within zsh's [[ ]] construct.
# - 'N' makes the glob pattern evaluate to nothing when it doesn't match (rather than throw a globbing error)
# - '.' matches "regular files"
# - 'mh+24' matches files (or directories or whatever) that are older than 24 hours.
#autoload -U compaudit compinit
#: ${ZSH_DISABLE_COMPFIX:=true}
# ...
setopt extendedglob
if [[ $ZSH_DISABLE_COMPFIX != true ]]; then
# If completion insecurities exist, warn the user without enabling completions.
if ! compaudit &>/dev/null; then
# This function resides in the "lib/compfix.zsh" script sourced above.
    handle_completion_insecurities
# Else, enable and cache completions to the desired file.
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
      compinit -d "${ZSH_COMPDUMP}"
      compdump
else
      compinit -C
fi
fi
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
    compinit -i -d "${ZSH_COMPDUMP}"
    compdump
else
    compinit -C
fi
fi

And I've loaded it through .zshrc in plugins=(zshfs git ...)

Thank you!

Did you mean plugins=(zshfl git ... ? :)

@divyekapoor
Copy link

divyekapoor commented Nov 6, 2020

Just check if the shell is an interactive shell before executing compinit?

eg. with [[ -o interactive ]] ?

@MostHated
Copy link

MostHated commented Mar 14, 2021

Skip the not really helping Ubuntu global compinit

skip_global_compinit=1

Just adding this one thing to .zshenv cut my initial loading time in half. 👍

For plugins, though. I use this: https://getantibody.github.io/ "The fastest shell plugin manager."

Plus, here are some additional tips to make initial loading faster if anyone is interested.
https://carlosbecker.com/posts/speeding-up-zsh/

(Man, I didn't realize it's already been 5 years since I came across that site. So I suppose its a bit of an old post, but most of it should still be relevant and helpful)

@smac89
Copy link

smac89 commented Nov 8, 2021

Over time, I've improved this snippet to become something like this:

function {
    # http://zsh.sourceforge.net/Doc/Release/Options.html#Scripts-and-Functions
    setopt LOCAL_OPTIONS extendedglob
    # http://zsh.sourceforge.net/Doc/Release/Expansion.html#Glob-Qualifiers
    # http://zsh.sourceforge.net/Doc/Release/Conditional-Expressions.html#Conditional-Expressions
    if [[ ! -e "${ZSH_COMPDUMP}" || -n "${ZSH_COMPDUMP}"(#qNY1.mh+24) ]]; then
        zi compinit > /dev/null
        zi cdreplay -q
    fi
}

A few notes:

  • ZSH_COMPDUMP comes from ohmyzsh, which I use in addition to zinit
  • The Y1 enables short-circuit mode so that only the first file which matches the rest of the pattern will be returned.
  • Everything is in an anonymous function so that we can enable extendedglob just for the function, and not for the entire shell

@aztack
Copy link

aztack commented Dec 14, 2021

autoload -Uz compinit
for dump in ~/.zcompdump(N.mh+24); do
  compinit
done
compinit -C

works for me.

Timing with timezsh:

timezsh() {
  shell=${1-$SHELL}
  for i in $(seq 1 4); do /usr/bin/time $shell -i -c exit; done
}

Before:

        1.33 real         0.80 user         0.52 sys
        1.38 real         0.80 user         0.57 sys
        1.55 real         0.82 user         0.64 sys
        1.50 real         0.81 user         0.63 sys

After:

        0.78 real         0.32 user         0.42 sys
        0.81 real         0.33 user         0.45 sys
        0.96 real         0.37 user         0.55 sys
        1.21 real         0.42 user         0.71 sys

@xinxilas
Copy link

xinxilas commented Mar 1, 2022

It was faster here

Definitive solution?

@vincentbernat
Copy link

vincentbernat commented Apr 7, 2022

I am going for

() {
    emulate -L zsh
    setopt extendedglob
    autoload -Uz compinit complist
    local zcd=$1                # compdump
    local zcdc=$1.zwc           # compiled compdump
    local zcda=$1.last          # last compilation
    local zcdl=$1.lock          # lock file
    local attempts=30
    while (( attempts-- > 0 )) && ! ln $zcd $zcdl 2> /dev/null; do sleep 0.1; done
    {
        if [[ ! -e $zcda || -n $zcda(#qN.mh+24) ]]; then
            compinit -i -d $zcd
            : > $zcda
        else
            compinit -C -d $zcd
        fi
        [[ ! -f $zcdc || $zcd -nt $zcdc ]] && rm -f $zcdc && zcompile $zcd &!
    } always {
        rm -f $zcdl
    }
} $ZSH/run/u/$HOST-$UID/zcompdump

I didn't see a mention what happens when multiple shells are spawned at the same time. Maybe that's the reason we sometime run into empty dump file.

@medwatt
Copy link

medwatt commented Apr 12, 2022

Where exactly do we put this code:

autoload -Uz compinit
for dump in ~/.zcompdump(N.mh+24); do
  compinit
done
compinit -C

@faelin
Copy link

faelin commented Jun 3, 2022

@medwatt put the snippet somewhere in your .zshrc file, or put it in an external script file and call it from your .zshrc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment