Skip to content

Instantly share code, notes, and snippets.

@donatello
Last active July 5, 2017 18:44
Show Gist options
  • Save donatello/116b8d8849c343f696794e0d96606e87 to your computer and use it in GitHub Desktop.
Save donatello/116b8d8849c343f696794e0d96606e87 to your computer and use it in GitHub Desktop.
Adding pattern matching to `mc`

Proposal for mc find command

mc find PATH [ACTION [print|exec ARGS...]]
  • PATH is either a file-path or a remote storage alias-path like myminio/mybucket
  • ACTION is a pattern matching expression and should be one of the following:
    • --name PATTERN e.g. --name '*.jpg' does glob match on just the base name of a file (should work like find command's -name option)
    • --path PATTERN e.g. --path '*/2014/*' glob matches on the whole path (should work like find command's -path option)
    • --regex PATTERN matches on the whole path with PCRE regular expression (note that find uses Emacs regular expressions by default, but we are going with PCRE).
    • --watch watches a path and performs actions as and when file events are received.
  • print command is used to substitute the found files within ARGS and print to stdout, e.g. mc find myminio/photos/2014/ --name '*.jpg' print '{base}' would print the base name of each matched file.
  • exec command spawns a process by executing ARGS per-file found, e.g. mc find myminio/photos/2014/ --name '*.jpg' exec "mc cp {} s3/backup/selfies/" copies each matching jpg files from myminio into the given path in s3.

In the last two points above, the string in curly braces are referred to as substitution arguments.

Substitution arguments can have the following variations:

  • {} -> prints the full path of the matched file relative to the input argument. E.g. mc find ./mydir/ print {} could print ./mydir/file1, and mc find myminio/bucket/ print {} could print myminio/bucket/file1.
  • {dir} -> prints the path to the file until the last forward slash (like UNIX dirname). It can be used to create some useful substitutions. E.g. mc find myminio/bucket/ print "{dir} has the file {}" could print a line like myminio/bucket/dir1 has the file myminio/bucket/dir1/file
  • {base} -> prints only the basename of a matched file (like UNIX basename).
  • Adding quotes inside the curly brackets causes the output to also have quotes like {"base"} would quote the outputted basename of the file. {"dirname"} and {""} would quote the dirname and the whole path respectively.

RAW NOTES (old)

mc find -> only lists files/objects (not directories or prefixes)

`--name PATTERN` -> match on basename and PATTERN is only a glob match (fnmatch(3))
`--path PATTERN` -> match on whole path and PATTERN is glob match (* matches anything)
`--regex PATTERN` -> PCRE regexp support
`--watch` -> watch paths


`--print`
`--exec`

# {} -> full path
# {dir} -> like dirname
# {"base"} -> like basename, with quotes
# {""} -> quoted full path

mc find myminio/photos/2014/ --name '*.jpg' print mc cp myminio/...

mc find PATH [PATTERN [print|exec ARGS...]]

mc find myminio/photos/2014/  print/exec [args]

mc find myminio/photos --json .. | jq -r .basename | xargs ?

{
  "basename": {base},
  "
}


a/b/c/d
{dir:1} -> b
{dir:-1} -> c
{dir} -> a/b/c/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment