Skip to content

Instantly share code, notes, and snippets.

@owahltinez
Last active October 6, 2021 02:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save owahltinez/40c88ee0bf4b2354782ef37a928c77a7 to your computer and use it in GitHub Desktop.
Save owahltinez/40c88ee0bf4b2354782ef37a928c77a7 to your computer and use it in GitHub Desktop.
Recursively list all files in an Azure container using `az storage fs file list`
import json
import os
import sys
import time
def list_files(opts: str, marker: str = None) -> list:
fpath = f"/tmp/{int(1E9 * time.monotonic())}.json"
marker_opt = f'--marker "{marker}"' if marker else ""
os.system(f'az storage fs file list {opts} --show-next-marker {marker_opt} > "{fpath}"')
with open(fpath) as fd:
data = json.load(fd)
marker = data[-1].get("nextMarker")
if marker:
data += list_files(opts, marker)
return data[:-1]
if __name__ == "__main__":
opts = " ".join(sys.argv[1:])
print(json.dumps(list_files(opts)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment