Speed up zsh compinit by only checking cache once a day.

compinit.zsh

# On slow systems, checking the cached .zcompdump file to see if it must be 
# regenerated adds a noticable delay to zsh startup.  This little hack restricts 
# it to once a day.  It should be pasted into your own completion file.
#
# The globbing is a little complicated here:
# - '#q' is an explicit glob qualifier that makes globbing work within zsh's [[ ]] construct.
# - 'N' makes the glob pattern evaluate to nothing when it doesn't match (rather than throw a globbing error)
# - '.' matches "regular files"
# - 'mh+24' matches files (or directories or whatever) that are older than 24 hours.
autoload -Uz compinit 
if [[ -n ${ZDOTDIR}/.zcompdump(#qN.mh+24) ]]; then
	compinit;
else
	compinit -C;
fi;

autoload -U compinit && (compinit &; compinit -C)

might work

A few notes about the this gist..
The file age test code seems to work even without #q or setopt extendedglob. Not quite sure why, but would need to check the zsh man page.

# 3 hours and older .zcompdump:
❯ [[ -n ${ZDOTDIR:-$HOME}/.zcompdump(N.mh+2) ]] &&  echo yes       
yes

Skipping Ubuntu system-wide compinit

Don't forget to check if the system wide zshrc is running compinit, and disable it if you can. So you can run your own compinit startup code.
Ubuntu does this in /etc/zsh/zshrc, and you can disable by adding this to your ~/.zshenv:

# Skip the not really helping Ubuntu global compinit
skip_global_compinit=1

Using Zinit - a fast Zsh plugin manager

Zinit provides a way to intercept compdef calls (used completion by completion scripts) to execute them later, after compinit, using the zinit cdreplay command:

# ~/.zshrc
source ~/.zinit/bin/zinit.zsh
zinit load "some/plugin"
zinit load "other/plugin"
autoload -Uz compinit; compinit
zinit cdreplay

More info:
https://github.com/zdharma/zinit#calling-compinit-without-turbo-mode

I am totally confused. Where is everyone adding these snippets? I have tried adding to my .zshrc to no success. The original gist mentions "zsh completion file" but what is that? Google turns up nothing.

I agree w/ @faaizajaz question; where do we put this stuff?

@faaizajaz, @duraki I can't speak with specifics about most of the additional discussion on this gist. (I no longer use any of these approaches) In general though, you are going to want to make your changes right after you load the compinit module. This is likely to be the file that compinstall created or edited.

To find this file on your system try zstyle | grep compinstall

Thanks @ctechols

This is how I did it.

I created this folder:

• mkdir ~/.oh-my-zsh/plugins/zshfl

Inside I've put:

• touch zshfl.plugin.zsh

# compinit optimization for oh-my-zsh
# On slow systems, checking the cached .zcompdump file to see if it must be
# regenerated adds a noticable delay to zsh startup.  This little hack restricts
# it to once a day.  It should be pasted into your own completion file.
#
# The globbing is a little complicated here:
# - '#q' is an explicit glob qualifier that makes globbing work within zsh's [[ ]] construct.
# - 'N' makes the glob pattern evaluate to nothing when it doesn't match (rather than throw a globbing error)
# - '.' matches "regular files"
# - 'mh+24' matches files (or directories or whatever) that are older than 24 hours.
#autoload -U compaudit compinit
#: ${ZSH_DISABLE_COMPFIX:=true}
# ...
setopt extendedglob
if [[ $ZSH_DISABLE_COMPFIX != true ]]; then
# If completion insecurities exist, warn the user without enabling completions.
if ! compaudit &>/dev/null; then
# This function resides in the "lib/compfix.zsh" script sourced above.
    handle_completion_insecurities
# Else, enable and cache completions to the desired file.
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
      compinit -d "${ZSH_COMPDUMP}"
      compdump
else
      compinit -C
fi
fi
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
    compinit -i -d "${ZSH_COMPDUMP}"
    compdump
else
    compinit -C
fi
fi

And I've loaded it through .zshrc in plugins=(zshfs git ...)

Thanks @ctechols

This is how I did it.

I created this folder:

• mkdir ~/.oh-my-zsh/plugins/zshfl

Inside I've put:

• touch zshfl.plugin.zsh

# compinit optimization for oh-my-zsh
# On slow systems, checking the cached .zcompdump file to see if it must be
# regenerated adds a noticable delay to zsh startup.  This little hack restricts
# it to once a day.  It should be pasted into your own completion file.
#
# The globbing is a little complicated here:
# - '#q' is an explicit glob qualifier that makes globbing work within zsh's [[ ]] construct.
# - 'N' makes the glob pattern evaluate to nothing when it doesn't match (rather than throw a globbing error)
# - '.' matches "regular files"
# - 'mh+24' matches files (or directories or whatever) that are older than 24 hours.
#autoload -U compaudit compinit
#: ${ZSH_DISABLE_COMPFIX:=true}
# ...
setopt extendedglob
if [[ $ZSH_DISABLE_COMPFIX != true ]]; then
# If completion insecurities exist, warn the user without enabling completions.
if ! compaudit &>/dev/null; then
# This function resides in the "lib/compfix.zsh" script sourced above.
    handle_completion_insecurities
# Else, enable and cache completions to the desired file.
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
      compinit -d "${ZSH_COMPDUMP}"
      compdump
else
      compinit -C
fi
fi
else
if [[ -n "${ZSH_COMPDUMP}"(#qN.mh+24) ]]; then
    compinit -i -d "${ZSH_COMPDUMP}"
    compdump
else
    compinit -C
fi
fi

And I've loaded it through .zshrc in plugins=(zshfs git ...)

Thank you!

Did you mean plugins=(zshfl git ... ? :)

Just check if the shell is an interactive shell before executing compinit?

eg. with [[ -o interactive ]] ?

Skip the not really helping Ubuntu global compinit

skip_global_compinit=1

Just adding this one thing to .zshenv cut my initial loading time in half. 👍

For plugins, though. I use this: https://getantibody.github.io/ "The fastest shell plugin manager."

Plus, here are some additional tips to make initial loading faster if anyone is interested.
https://carlosbecker.com/posts/speeding-up-zsh/

(Man, I didn't realize it's already been 5 years since I came across that site. So I suppose its a bit of an old post, but most of it should still be relevant and helpful)

Over time, I've improved this snippet to become something like this:

function {
    # http://zsh.sourceforge.net/Doc/Release/Options.html#Scripts-and-Functions
    setopt LOCAL_OPTIONS extendedglob nullglob
    # http://zsh.sourceforge.net/Doc/Release/Expansion.html#Glob-Qualifiers
    # http://zsh.sourceforge.net/Doc/Release/Conditional-Expressions.html#Conditional-Expressions
    if [[ ! -e "${ZSH_COMPDUMP}" || -n ${~"${ZSH_COMPDUMP}"}(#qmh+24) ]]; then
        zi compinit > /dev/null
        zi cdreplay -q
    fi
}

---

A few notes:

• zi comes from zinit
• ZSH_COMPDUMP comes from ohmyzsh, which I use in addition to zinit
• The ${~...} syntax is to enable glob expansion with braces because the name of the file is in a variable
• Everything is in an anonymous function so that we can enable extendedglob just for the function, and not for the entire shell

autoload -Uz compinit
for dump in ~/.zcompdump(N.mh+24); do
  compinit
done
compinit -C

works for me.

Timing with timezsh:

timezsh() {
  shell=${1-$SHELL}
  for i in $(seq 1 4); do /usr/bin/time $shell -i -c exit; done
}

Before:

        1.33 real         0.80 user         0.52 sys
        1.38 real         0.80 user         0.57 sys
        1.55 real         0.82 user         0.64 sys
        1.50 real         0.81 user         0.63 sys

After:

        0.78 real         0.32 user         0.42 sys
        0.81 real         0.33 user         0.45 sys
        0.96 real         0.37 user         0.55 sys
        1.21 real         0.42 user         0.71 sys

It was faster here

Definitive solution?

I am going for

() {
    emulate -L zsh
    setopt extendedglob
    autoload -Uz compinit complist
    local zcd=$1                # compdump
    local zcdc=$1.zwc           # compiled compdump
    local zcda=$1.last          # last compilation
    local zcdl=$1.lock          # lock file
    local attempts=30
    while (( attempts-- > 0 )) && ! ln $zcd $zcdl 2> /dev/null; do sleep 0.1; done
    {
        if [[ ! -e $zcda || -n $zcda(#qN.mh+24) ]]; then
            compinit -i -d $zcd
            : > $zcda
        else
            compinit -C -d $zcd
        fi
        [[ ! -f $zcdc || $zcd -nt $zcdc ]] && rm -f $zcdc && zcompile $zcd &!
    } always {
        rm -f $zcdl
    }
} $ZSH/run/u/$HOST-$UID/zcompdump

I didn't see a mention what happens when multiple shells are spawned at the same time. Maybe that's the reason we sometime run into empty dump file.

Where exactly do we put this code:

autoload -Uz compinit
for dump in ~/.zcompdump(N.mh+24); do
  compinit
done
compinit -C

@medwatt put the snippet somewhere in your .zshrc file, or put it in an external script file and call it from your .zshrc

@vincentbernat I solved the multiple shell problem by using a lockfile and a trap to remove it. This ensures that only one process is responsible for updating the file. Even though there's still a race condition, it's now almost impossibly small ¹

() {
	setopt local_options
	setopt extendedglob

	local zcd=${1}
	local zcomp_hours=${2:-24} # how often to regenerate the file
	local lock_timeout=${2:-1} # change this if compinit normally takes longer to run
	local lockfile=${zcd}.lock

	if [ -f ${lockfile} ]; then 
		if [[ -f ${lockfile}(#qN.mm+${lock_timeout}) ]]; then
			(
				echo "${lockfile} has been held by $(< ${lockfile}) for longer than ${lock_timeout} minute(s)."
				echo "This may indicate a problem with compinit"
			) >&2 
		fi
		# Exit if there's a lockfile; another process is handling things
		return
	else
		# Create the lockfile with this shell's PID for debugging
		echo $$ > ${lockfile}
		# Ensure the lockfile is removed
		trap "rm -f ${lockfile}" EXIT
	fi

	autoload -Uz compinit

	if [[ -n ${zcd}(#qN.mh+${zcomp_hours}) ]]; then
		# The file is old and needs to be regenerated
		compinit
	else
		# The file is either new or does not exist. Either way, -C will handle it correctly
		compinit -C
	fi
} ${ZDOTDIR:-$HOME}/.zcompdump 

---

For those who are new to some of this stuff, check out

• https://zsh.sourceforge.io/Doc/Release/Completion-System.html
  • For how compinit works
• https://zsh.sourceforge.io/Doc/Release/Expansion.html#Filename-Generation
  • For the (#q.N.....) syntax, search for "file access qualifier"

Footnotes

1. A second process would have to check for the file in between 2 (effectively) subsequent commands. if [ -f ${lockfile} ] and echo $$ > ${lockfile} ↩

@thefotios my solution is already handling this case (but my message wasn't quite clear on that, so I understand you thought this was not the case). This is the purpose of the ln command (which is atomic, so no race condition).

I commented out these lines in my .zshrc which sped it up a lot:

if type brew &>/dev/null; then
    FPATH=$(brew --prefix)/share/zsh-completions:$FPATH

    autoload -Uz compinit
    compinit
if

As this comment above pointed out, oh-my-zsh already runs compinit.

@aztack Thank you for your snippet, it helps alot :)

Fellas, if you're of that kind that checks their .zsh files and scripts with shellcheck (like I am), here's a more POSIX-compliant (as much as zsh allows) statement:

if [ "$(find ~/.zcompdump -mtime 1)" ] ; then
    compinit
fi
compinit -C

or oneliner, if you prefer that:

# negation, so that at least one exits on 0
[ ! "$(find ~/.zcompdump -mtime 1)" ] || compinit
compinit -C

find's manpage is not clear on this, but I believe you want -mtime +1 to catch a file at least 24h old, rather than exactly 24h old.
On OSX, my .zcomdump was several days old, and the above would not trigger until I added the '+'.

@aztack It helped. 🙇

This is my take on the problem, it's a tradeoff between efficiency and simplicity:

autoload -Uz compinit; compinit -C  # Use cache to reduce startup time by ~0.1s
# Have another thread refresh the cache in the background (subshell to hide output)
(autoload -Uz compinit; compinit &)

Despite the obvious pitfall (having the shell start another thread at startup), I wonder if it's overall a good solution 🤔

Here is an update that combines everyone's suggestions into one function! Add this to your .zshrc for a fast, (relatively) thread-safe load of your zsh completions! (I left out the zinit suggestion shared above, to keep this more universal)

() {
  setopt local_options

  local zcompdump="${ZDOTDIR:-$HOME}/.zcompdump"
  local zcomp_ttl=1  # how many days to let the zcompdump file live before it must be recompiled
  local lock_timeout=1  # register an error if lock-timeout exceeded
  local lockfile="${zcompdump}.lock"

  autoload -Uz compinit

  # check for lockfile — if the lockfile exists, we cannot run a compinit
  #   if no lockfile, then we will create one, and set a trap on EXIT to remove it;
  #   the trap will trigger after the rest of the function has run.
  if [ -f "${lockfile}" ]
  then 

    # error log if the lockfile outlived its timeout
    if [ "$( find "${lockfile}" -mmin $lock_timeout )" ]
    then
      (
        echo "${lockfile} has been held by $(< ${lockfile}) for longer than ${lock_timeout} minute(s)."
        echo "This may indicate a problem with compinit"
      ) >&2
    fi

    # since the zcompdump is still locked, run compinit without generating a new dump
    compinit -D -d "$zcompdump"

    # Exit if there's a lockfile; another process is handling things
    return 1

  else

    # Create the lockfile with this shell's PID for debugging
    echo $$ > "${lockfile}"

    # Ensure the lockfile is removed on exit
    trap "rm -f '${lockfile}'" EXIT

  fi


  # refresh the zcompdump file if needed
  if [ ! -f "$zcompdump" -o "$( find "$zcompdump" -mtime "+${zcomp_ttl}" )" ]
  then
    # if the zcompdump is expired (past its ttl) or absent, we rebuild it
    compinit -d "$zcompdump"

  else

    # load the zcompdump without updating
    compinit -CD -d "$zcompdump"

    # asynchronously rebuild the zcompdump file
    (autoload -Uz compinit; compinit -d "$zcompdump" &);

  fi
}

Follow-on to the above, I ran a benchmark using hyperfine and found that a regular compinit is still faster...

Just to be extra fair I compared three versions: standard compinit; the script I shared directly above this message; same as the script above (compinit_subshells.zsh), but using the extendedglob syntax suggested by @thefotios above instead of the subshells (compinit_fast.zsh).

❯ hyperfine --show-output --shell='zsh -l' --warmup 3 --min-runs 30 --setup 'autoload -Uz compinit; compinit;' './compinit_fast.zsh' './compinit_subshells.zsh' 'autoload -Uz compinit; compinit'
Benchmark 1: ./compinit_fast.zsh
  Time (mean ± σ):      70.5 ms ±  40.0 ms    [User: 0.0 ms, System: 0.0 ms]
  Range (min … max):    18.1 ms … 191.8 ms    30 runs

Benchmark 2: ./compinit_subshells.zsh
  Time (mean ± σ):      89.1 ms ±  50.7 ms    [User: 0.0 ms, System: 0.0 ms]
  Range (min … max):    14.3 ms … 229.7 ms    30 runs

Benchmark 3: autoload -Uz compinit; compinit
  Time (mean ± σ):      59.1 ms ±  31.1 ms    [User: 45.0 ms, System: 17.2 ms]
  Range (min … max):    16.4 ms … 119.3 ms    30 runs

Summary
  autoload -Uz compinit; compinit ran
    1.19 ± 0.92 times faster than ./compinit_fast.zsh
    1.51 ± 1.17 times faster than ./compinit_subshells.zsh                                                                                                            /1m18.1s


❯ hyperfine --show-output --shell='zsh' --warmup 3 --min-runs 30 --setup 'autoload -Uz compinit; compinit;' './compinit_fast.zsh' './compinit_subshells.zsh' 'autoload -Uz compinit; compinit'
Benchmark 1: ./compinit_fast.zsh
  Time (mean ± σ):      69.1 ms ±   6.2 ms    [User: 38.9 ms, System: 23.8 ms]
  Range (min … max):    61.2 ms …  84.4 ms    32 runs

Benchmark 2: ./compinit_subshells.zsh
  Time (mean ± σ):      75.3 ms ±   5.6 ms    [User: 39.5 ms, System: 26.8 ms]
  Range (min … max):    68.3 ms …  90.2 ms    34 runs

Benchmark 3: autoload -Uz compinit; compinit
  Time (mean ± σ):      58.7 ms ±   4.0 ms    [User: 44.2 ms, System: 14.3 ms]
  Range (min … max):    53.1 ms …  74.5 ms    43 runs

Summary
  autoload -Uz compinit; compinit ran
    1.18 ± 0.13 times faster than ./compinit_fast.zsh
    1.28 ± 0.13 times faster than ./compinit_subshells.zsh                                                                                                              /10.7s

This even includes using antidote to load a bunch of plugins, so I know there are a lot of completions to load.

So at this point, the above "fast" implementation may no longer hold value in recent zsh builds. Please feel free to share your own benchmarks, though in case it's useful!

For oh-my-zsh users, I wrote a bit to integrate this check into oh-my-zsh. At some point, I'll submit a PR to omz: ohmyzsh/ohmyzsh@master...forivall:oh-my-zsh:use-cached-compdump

From 3514d7099b68a06659f4882adfff808ab6fa0d51 Mon Sep 17 00:00:00 2001
From: Emily M Klassen <forivall@gmail.com>
Date: Fri, 1 Nov 2024 14:12:56 -0700
Subject: [PATCH] feat(completions): add option to use a cached compdump

---
 oh-my-zsh.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/oh-my-zsh.sh b/oh-my-zsh.sh
index b1032841c677..447b006f4344 100644
--- a/oh-my-zsh.sh
+++ b/oh-my-zsh.sh
@@ -121,7 +121,11 @@ if ! command grep -q -Fx "$zcompdump_revision" "$ZSH_COMPDUMP" 2>/dev/null \
   zcompdump_refresh=1
 fi
 
-if [[ "$ZSH_DISABLE_COMPFIX" != true ]]; then
+if [[ "$ZSH_COMPINIT_CACHE" == true && ! (( $zcompdump_refresh )) ]] \
+   && () { setopt local_options extendedglob; [[ -z "$ZSH_COMPDUMP"(#qN.mh+24) ]] }; then
+  # If the compdump was modified less than 24 hours ago, use the cached compdump, disable autodump
+  compinit -C -d "$ZSH_COMPDUMP" -D
+elif [[ "$ZSH_DISABLE_COMPFIX" != true ]]; then
   source "$ZSH/lib/compfix.zsh"
   # Load only from secure directories
   compinit -i -d "$ZSH_COMPDUMP"

From my ad-hoc testing, this saves about 50ms on startup, on my dotfiles setup, which loads omz through zgenom