Skip to content

Instantly share code, notes, and snippets.

@roalcantara
Last active December 9, 2023 00:30
Show Gist options
  • Star 10 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save roalcantara/5cf137f83261d8108f53019748c17222 to your computer and use it in GitHub Desktop.
Save roalcantara/5cf137f83261d8108f53019748c17222 to your computer and use it in GitHub Desktop.
Glob (globbing)
## TL;DR
setopt extendedglob
ls *(<tab> # to get help regarding globbing
rm ../debianpackage(.) # remove files only
ls -d *(/) # list directories only
ls /etc/*(@) # list symlinks only
ls -l *.(png|jpg|gif) # list pictures only
ls *(*) # list executables only
ls /etc/**/zsh # which directories contain 'zsh'?
ls **/*(-@) # list dangling symlinks ('**' recurses down directory trees)
ls foo*~*bar* # match everything that starts with foo but doesn't contain bar
ls *(e:'file $REPLY | grep -q JPEG':) # match all files of which file says that they are JPEGs
ls -ldrt -- *(mm+15) # List all files older than 15mins
ls -ldrt -- *(.mm+15) # List Just regular files
ls -ld /my/path/**/*(D@-^@) # List the unbroken sysmlinks under a directory.
ls -Lldrt -- *(-mm+15) # List the age of the pointed to file for symlinks
ls -l **/README # Search for `README' in all Subdirectories
ls -l foo<23-> # List files beginning at `foo23' upwards (foo23, foo24, foo25, ..)
ls -l 200406{04..10}*(N) # List all files that begin with the date strings from June 4 through June 9 of 2004
ls -l 200306<4-10>.* # or if they are of the form 200406XX (require ``setopt extended_glob'')
ls -l *.(c|h) # Show only all *.c and *.h - Files
ls -l *(R) # Show only world-readable files
ls -fld *(OL) # Sort the output from `ls -l' by file size
ls -fl *(DOL[1,5]) # Print only 5 lines by "ls" command (like ``ls -laS | head -n 5'')
ls -l *(G[users]) # Show only files are owned from group `users'
ls *(L0f.go-w.) # Show only empty files which nor `group' or `world writable'
ls *.c~foo.c # Show only all *.c - files and ignore `foo.c'
print -rl /home/me/**/*(D/e{'reply=($REPLY/*(N[-1]:t))'}) # Find all directories, list their contents and output the first item in the above list
print -rl /**/*~^*/path(|/*) # Find command to search for directory name instead of basename
print -l ~/*(ND.^w) # List files in the current directory are not writable by the owner
print -rl -- *(Dmh+10^/) # List all files which have not been updated since last 10 hours
print -rl -- **/*(Dom[1,10]) # List the ten newest files in directories and subdirs (recursive)
print -rl -- /path/to/dir/**/*(D.om[5,10]) # Display the 5-10 last modified files
print -rl -- **/*.c(D.OL[1,10]:h) | sort -u # Print the path of the directories holding the ten biggest C regular files in the current directory and subdirectories.
setopt dotglob ; print directory/**/*(om[1]) # Find most recent file in a directory
for a in ./**/*\ *(Dod); do mv $a ${a:h}/${a:t:gs/ /_}; done # Remove spaces from filenames
# * (#s) or (#e) for what ^ and $ are in regexps (beginning of line/end of line)
# * (#b) or (#m) to enable backreferences
# * (#i) to match case insensitive
# * (#a) to match approximately (certain errors are ignored, e.g. “(#a1)foo*” matches the string “ofobar”)
# sources:
# * https://github.com/mika/zsh-pony#globbing--glob-qualifiers
# * https://strcat.de/zsh/#tipps
## Globbing
# A glob is a short expression that lets you filter files by their name
# 99% of the time, there’s an asterisk involved.
## Given the following structure
#
# zsh_demo
# ├── data
# │ ├── africa
# │ │ ├── kenya
# │ │ │ ├── literacy.txt
# │ │ │ ├── income.txt
# │ │ │ └── population.txt
# │ │ └── ...
# │ ├── asia
# │ │ ├── ...
# │ └── europe
# │ ├── ...
# └── data
# ├── africa
# │ ├── kenya
# │ │ ├── literacy_index.txt
# │ │ ├── median_income.txt
# │ │ └── population_by_province.txt
# │ └── ...
# ├── ...
ls zsh_demo/**/*.txt # <= this is a glob
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### 1. File picking
# Globs get replaced by the names of the files that match the glob expression.
ls zsh_demo # list every file directly below the zsh_demo folder
ls zsh_demo/* # list every file in the folders directly below the zsh_demo folder
ls zsh_demo/*/* # list every file in every folder two levels below the zsh_demo folder
ls zsh_demo/**/* # list every file anywhere below the zsh_demo folder
ls zsh_demo/**/*.txt # list every file that ends in .txt in every folder at any level below the zsh_demo folder
print -l zsh_demo/data/*/* # list folders separated by new lines
echo zsh_demo/data/*/* # list folders separated by spaces
for country_folder in zsh_demo/data/*/*; do # for each folder
# do something
done
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
## 1.1 Glob Operators
# Filter by file names
ls -l zsh_demo/**/*<1-10>.txt # list text files that end in a number from 1 to 10
ls -l zsh_demo/**/[a]*.txt # list text files that start with the letter a
ls -l zsh_demo/**/(ab|bc)*.txt # list text files that start with either ab or bc
ls -l zsh_demo/**/[^cC]*.txt # list text files that don't start with a lower or uppercase c
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
# more: http://zsh.sourceforge.net/Doc/Release/Expansion.html#Filename-Generation
## 2.1 Glob Qualifiers
# Filter by file type, size, modification date
# Glob qualifiers are surrounded in parentheses (), and appear at the end of a glob to make it more stringent.
print -l zsh_demo/**/*(/) # show only directories
print -l zsh_demo/**/*(.) # show only regular files
ls -l zsh_demo/**/*(L0) # show empty files
ls -l zsh_demo/**/*(Lk+3) # show files greater than 3 KB
print -l zsh_demo/**/*(mh-1) # show files modified in the last hour
ls -l zsh_demo/**/*(om[1,3]) # sort files from most to least recently modified and show the last 3
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
## 2.1 Glob Qualifiers
# To FIND every continent that DOES NOT CONTAIN a country named malta
#
print -l zsh_demo/*/*(e:'[[ ! -e $REPLY/malta ]]':)
# 1. After the e, the string has to be delimited by a convenient character (in this case, a colon :),
# and the code must be surrounded by single quotes ',
# so the actual command is just [[ ! -e $REPLY/malta ]].
#
# 2. The $REPLY variable contains every file name of the ones specified by the glob zsh_demo/*/* in turn,
# but only a single file at a time.
#
# 3. [[ -e file ]] is a conditional expression that returns true if the file exists.
# We want it to return true when the file called malta doesn’t exist, so we reverse it with !.
#
# 4. When the code is executed, the $REPLY variable takes the value of the next file and the code is executed again.
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
## 2.1 Glob Qualifiers
# To SORT files from most to least recently modified and show the last 3
#
ls -l zsh_demo/**/*(om[1,3])
# Five different things are going on at the same time:
# 1. The . tells the glob to only show regular files (no directories, symbolic links, or other types of files).
#
# 2. The Lm-2 tells the glob to show files smaller than 2 MB.
# * Use - for smaller, and + for greater; don’t use anything if you want to specify the exact size (Lm2).
# * Use m for megabytes, k for kilobytes, or nothing for just bytes (notice that these letters must appear before the sign).
#
# 3. The mh-1 tells the glob to show files modified in the last hour
# * Use - if you want files modified within the last X units of time, and + for files modified more than X units of time ago.
# * Use M for Months, w for weeks, h for hours, m for minutes, and s for seconds (notice that these leters must appear before the sign).
#
# 4. The om tells the glob to sort the remaining files by their modification date.
# * A lowercase o sorts by most recent first, to use the reverse order, make it uppercase O.
# * Use m to sort by modification date, and L to sort by size (oL).
#
# 5. The [1,3] tells the glob to show the first 3 files (since we just sorted the files, these will be the most recently modified ones).
# * You can also show a single file (for example, the second one [2])
# more: http://zsh.sourceforge.net/Doc/Release/Expansion.html#Filename-Generation
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### DELETE examples
## delete only the oldest file in a directory
rm ./*filename*(Om[1])
## Find and delete the files which are older than a given parameter (seconds/minutes/hours)
#
rm -f /Dir/**/*(.mh+3) # deletes all regular file in /Dir that are older than 3 hours
rm -f /Dir/**/*(@mm+3) # deletes all symlinks in /Dir that are older than 3 minutes
rm -f /Dir/**/*(ms+30^/) # deletes all non dirs in /Dir that are older than 30 seconds
rm ./**/*(.Dmh+1,.DL0) # deletes all folders, sub-folders and files older than one hour
rm -f **/*(mh+6) # deletes all files more than 6 hours old
rm ./*(Om[1,-11]) # removes all files but the ten newer ones (delete all but last 10 files in a directory)
## remove empty directories afterwards
rmdir ./**/*(/od) 2> /dev/null
## Note: If you get a arg list too long, you use the builtin rm. For example:
zmodload zsh/files ; rm -f **/*(mh+6)
# or use the zargs function:
autoload zargs ; zargs **/*(mh+6) -- rm -f
# source: https://strcat.de/zsh/#tipps
### CHMOD examples
## Recursive chmod
chmod 700 **/(.) # Only files
chmod 700 **/(/) # Only directories
chown 666 **/*(u102) # Change the UID from 102 to 666
## find all files without a valid owner
chmod someuser /**/*(D^u:${(j.:u:.)${(f)"$(</etc/passwd)"}%%:*}:)
## Search all files in /home/*/*-mail/ with a setting ``chmod -s'' flag
# (recursive, include dotfiles) remove the setgid/setuid flag and print a message
chmod -s /home/*/*-mail(DNs,S) /home/*/*-mail/**/*(DNs,S))
# source: https://strcat.de/zsh/#tipps
### FIND examples
## find directories that contain both "index.php" and "index.html", or in general, directories
# that contain more than one file matching "index.*"
ls **/*(D/e:'[[ -e $REPLY/index.php && -e $REPLY/index.html ]]':)
## Find files with size == 0 and send a mail
files=(**/*(ND.L0m+0m-2)) > (( $#files > 0 )) && print -rl -- $files | mailx -s "empty files" foo@bar.tdl
## Print out all of the files in that directory in 2 columns
print -rC2 -- ${1:[...]}/*(D:t)
# ^- number ob columns
# or - if you feel concerned about special characters - use
list=(${1:[...]}/*(ND:t))
(($#list)) && print -rC2 -- ${(V)list}
## Show data to *really* binary format
zsh -ec 'while {} {printf %.8x $n;repeat 8 \
> {read -ku0 a printf \ %.8d $(([##2]#a))};print;((n+=8))}' < binary
## A User's Guide to the Z-Shell /5.9: Filename Generation and Pattern Matching
# find all files in all subdirectories, searching recursively, which have a given
# name, case insensitive, are at least 50 KB large, no more than a week old and
# owned by the root user, and allowing up to a single error in the spelling of
# the name. In fact, the required expression looks like this:
ls **/(#ia1)name(LK+50mw-1u0)
## find all the empty directories in a tree
for f in ***/*(/l2); do foo=($f/*(N)); [[ -z $foo ]] && print $f; done
# Note:Since Zsh 4.2.1(?) the glob qualifier F indicates a non-empty directory.
Hence *(F) indicates all subdirectories with entries, *(/^F) means all subdirectories with no
entries.
ls -ld *(/^F)
## Use find(1) to find all directories except the ".svn" ones,
# then use grep on all *.c/*.h/*.S files in each directory.
setopt extendedglob
grep pattern (^.svn/)#*.[xhS](.) # skips dot files and dot directories
# or - without Zsh
find . -type d ! -name .svn | sed 's/./\\&/g;s|.*|grep whatever &/*.[xhS]|' | sh
## Quote from Usenet
# > I need to write a script that searches through a folder on my Linux
# > home server and copies only files where the width is greater than the
# > height, but it also needs to flatten the directory structure and
# > rename files as it copies. So the first picture found may be several
# > directories deep in the source but this should be copied to the root
# > of the SD card and named say 00000001.JPG and the next file found
# > where width > height should be copied and renamed 00000002.JPG.
width_greater_than_height() {
local w h
identify -format '%w %h' ${1-$REPLY} |
read w h && ((w > h))
}
typeset -Z8 i=0
setopt extendedglob
for f (**/*.(#i)jp(e|)g(D.+width_greater_than_height)) {
((i++))
cp -- $f /path/to/dest/$i.JPG
}
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
# Inside the parentheses, each modifiers is preceded by a colon :, which makes them easily distinguishable from qualifiers.
print -l zsh_demo/data/europe/poland/*.txt # A plain old glob
print -l zsh_demo/data/europe/poland/*.txt(:t) # Return the file name (t stands for tail)
print -l zsh_demo/data/europe/poland/*.txt(:t:r) # Return the file name without the extension (r stands for remove_extension, I think)
print -l zsh_demo/data/europe/poland/*.txt(:e) # Return the extension
print -l zsh_demo/data/europe/poland/*.txt(:h) # Return the parent folder of the file (h stands for head)
print -l zsh_demo/data/europe/poland/*.txt(:h:h) # Return the parent folder of the parent
print -l zsh_demo/data/europe/poland/*.txt([1]:h) # Return the parent folder of the first file
# Remember you can combine qualifiers and modifiers.
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### 2. Variable transformations: Parameter expansion
# Modifiers are not only for globs, you can also use them with variables (the technical term is parameter expansion):
# If you want to store a glob in a variable, you must use parentheses
my_file=(zsh_demo/data/europe/poland/*.txt([1]))
print -l $my_file
print -l $my_file(:h) # this is the syntax we saw before
print -l ${my_file:h} # I find this syntax more convenient
print -l ${my_file(:h)} # DO NOT mix the two, or you'll get an ERROR!
print -l ${my_file:u} # the :u modifier makes the text uppercase
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### 2. Variable transformations: Modifiers
#
## :e(xtension)
foo=23.42 # Remove all but the extension
echo $foo:e
# -> 42
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
#
# The `:gs` modifier (g stands for global)
# Used to more than one substitution
my_variable="aaa"
echo ${my_variable:s/a/A/}
# -> Aaa
echo ${my_variable:gs/a/A/}
# -> AAA
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### 2. Variable transformations: Modifiers
#
# (:h)ead - like `dirname'.
echo =ls(:h) # Remove a trailing pathname component, leaving the head.
# -> /bin
echo =ls(:h) # Print the new command but do not execute it. Only works with history expansion.
# -> /bin
!echo:p
echo =ls(:h)
for f (*.sh) mv $f $f:h # Remove the suffix from each file (*.sh in this example)
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
#
# (:l)owercase
bar=FOOBAR
echo $bar:l # Convert the words to all lowercase.
# -> foobar
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
foo="one two three four"
print -r -- "${(C)var}" # convert 1st char of a word to uppercase
# -> One Two Three Four
a=( a a f 1 3 b b 3 5 4 4 ) # Eliminate the duplicated elements of an array
a=("${(u@)a") # and keep the remain emements order appeared in the original
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
#
# (:q)uote
bar="23'42"
echo $bar:q # Quote the substituted words, escaping further substitutions.
# -> 23\'42
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
#
# :r(rest)
for f (*.sh) mv $f $f:r # Remove the suffix from each file (*.sh in this example)
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
#
# The `:s` modifier
#
# Let’s say we wanted to calculate the maximum income for each country,
# and store it in a file named {country}_max_income.txt in the corresponding calculations folder.
#
# We can do this easily using the modifier (:s):
# Each time the for loop runs, the $file variable is set to a different income file:
for file in zsh_demo/data/**/income.txt ; do
# We use the :h modifier to get rid of the file name (zsh_demo/data/africa/kenya/),
# and then we use the :s modifier to substitute data with calculations: (zsh_demo/calculations/africa/kenya/)
output_dir=${file:h:s/data/calculations/}
# We use the :t modifier to get the name of the country (kenya)
country=${output_dir:t}
# Then we stick a slash / between the $output_dir and $country variables,
# and append _max_income.txt to get our output file path
# (ex: zsh_demo/calculations/africa/kenya/kenya_max_income.txt)
output_file="${output_dir}/${country}_max_income.txt"
# The $RANDOM variable gives you a random number every time you call it
# (just a quick way of generating some content).
# The right arrow > saves the calculation to the output file.
echo "The max salary is $RANDOM dollars" > $output_file
done
# The `grep "" bunch_of_files` command is a quick-and-dirty way to show the name of each file and its contents
# we could have also used `head bunch_of_files`
grep "" zsh_demo/calculations/**/*_max_income.txt
#
# You can use any character to separate the :s and the strings:
#
my_variable="path/abcd"
echo ${my_variable:s/bc/BC/} # path/aBCd
echo ${my_variable:s_bc_BC_} # path/aBCd
## escaping the slash \/
echo ${my_variable:s/\//./}
# -> path.abcd
## to substitute the slash (/) without escape
echo ${my_variable:s_/_._}
# -> # path.abcd
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### 2. Variable transformations: Modifiers
#
# :t(tail) - like `basename'.
foo=/usr/src/linux
echo $foo:t # Remove a filename extension of the form `.xxx', leaving the root name.
# -> linux
echo =ls(:t) # Remove all leading pathname components, leaving the tail.
# -> ls
for f (*.sh) mv $f $f:t # Remove the suffix from each file (*.sh in this example)
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Modifiers
#
# :u(ppercase)
bar=foobar
echo $bar:u # Convert the words to all uppercase.
# -> FOOBAR
# source: https://strcat.de/zsh/#tipps
### 2. Variable transformations: Expansion flags
#
# The s(plit) flag
# Let's say somebody gave you these updated files
# and told you to replace the old ones
echo $RANDOM > zsh_demo/africa_malawi_population_2014.txt
echo $RANDOM > zsh_demo/asia_nepal_income_2014.txt
echo $RANDOM > zsh_demo/europe_malta_literacy_2014.txt
# How would you move them to their appropriate folders?
# for each $file (ex: zsh_demo/europe_malta_literacy_2014.txt)
for file in zsh_demo/*.txt; do
# 1. Use the :t modifier to get rid of everything to the left of the first slash /.
# $ echo ${file:t}
# $ -> europe_malta_literacy_2014.txt
#
# 2. Use the (s) expansion flag to split the file name at each underscore _.
# $ echo ${(s._.)file:t}
# $ -> europe malta literacy 2014.txt
#
# 3. Without parentheses, file_info contains the wrong information:
# # file_info=${(s._.)file:t}
# $ echo $file_info
# $ -> europe_malta_literacy_2014.txt
#
file_info=(${(s._.)file:t}) # So, surround everything with parentheses
# $ echo $file_info
# $ -> europe malta literacy 2014.txt
# 5. Use an auxiliary variable for continent, country, and data.
# Since $file_info is now an array, we can refer to its elements by using a numeric index.
# $ echo ${file_info[3]}
# $ -> literacy
continent=$file_info[1]
country=$file_info[2]
data=$file_info[3]
# 6. Use the auxiliary variables to specify the path where we want to move the new files
mv -f $file zsh_demo/data/${continent}/${country}/${data}.txt
done
# Check the contents of the files (.) modified (m) in the last
# 5 minutes (m-5) to see what you just did
grep "" zsh_demo/**/*(.mm-5)
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
# more: http://zsh.sourceforge.net/Doc/Release/Expansion.html#Parameter-Expansion
### 2. Variable transformations: Expansion flags
#
# The j(oin) flag:
# Which does the opposite of the split flag
my_array=(a b c d)
echo ${(j.-.)my_array}
# -> a-b-c-d
# Since we are joining using dots (.), it makes more sense to
# use underscores (_) to separate the dots and the j
echo ${(j_._)my_array}
# -> a.b.c.d
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### 3. Magic tabbing: Event designators
#
# An event designator references one of the commands that we have previously entered.
# They always start with a bang !:
# show the previous command
echo a b c
!! # instead of pressing <Enter>, press <Tab>, then press <Enter>
# show two commands ago
echo d e f
echo g h i
!-2 # press <Tab>, then press <Enter>
# Note: If you press <Enter> instead of <Tab>,
# the event designator will also get replaced,
# but you’ll still have to press <Enter> one more time to run it.
# They really come in handy is to add previous arguments to our current command.
# add the last argument
ls zsh_demo/data/asia/laos/population.txt
ls -l !!1 # press <Tab>, then press <Enter>
# add all the previous arguments
echo a b c
print -l !!* # press <Tab>, then press <Enter>
# So, we reference previous arguments in two steps:
# 1. Specify which command you are interested in
# * The previous command !! is the one you’ll use most often.
# * If you want to go back farther, use the minus sign - and a number: !-2, !-3.
# * You can also use the current command !#
# 2. Pick what arguments you want to reuse
# * To pick an argument from the previous command,
# just add a number !!1, !!2. Use !!$ for the last argument.
# * To pick an argument from two or more commands ago,
# add a colon : before the number !-2:1 (because!-21 means something else).
# * If you want to reference all the arguments, use an asterisk * !!* !-2:*.
# * If you want skip all the arguments except the first one or two,
# add a number before the asterisk !!2*, !-2:2*.
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
### 3. Magic tabbing: Event designators
#
# Some useful examples
mv zsh_demo/data/asia/laos/population.txt !#1
# press <Tab>
# now you can easily change the second argument
# (use Control W to delete every up to the first slash)
ls zsh_demo/data/europe/malta/literacy.txt
awk '$1 > 3' !$
# press <Tab>
# !$ is a shortcut for !!$
ls zsh_demo/*/*/nepal/literacy.txt
ls zsh_demo/*/*/malta/literacy.txt
ls -l !-2:1
# press <Tab>
# now you can see the details of the nepal file
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
# more: http://zsh.sourceforge.net/Doc/Release/Expansion.html#History-Expansion
### 3. Magic tabbing: Event designators
#
# Pressing <Tab> lets you expand not only old commands, but globs,
# variables (when they use the ${} syntax), and even lazily-typed paths!
ls zsh_demo/*/*/nepal/literacy.txt
# press <Tab>
my_var="1 2 3"
echo ${my_var}
# press <Tab>
ls z/d/a/l
# press <Tab>
# 🤯
# source: http://reasoniamhere.com/2014/01/11/outrageously-useful-tips-to-master-your-z-shell/
# more: http://zsh.sourceforge.net/Doc/Release/Expansion.html#History-Expansion
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment