Skip to content

Instantly share code, notes, and snippets.

@denis-bz
Created July 10, 2018 12:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save denis-bz/fea4646a159fc43e9a4cf07bf65d28d5 to your computer and use it in GitHub Desktop.
Save denis-bz/fea4646a159fc43e9a4cf07bf65d28d5 to your computer and use it in GitHub Desktop.
Gish: sharing files on gist.github by name 10jul

Gish: sharing files on gist.github by name

Keywords: file sharing, gist, github, CLI, python, remote file server

gish is a command-line program to copy files between local computers and gist.github.com, using file names or gist ids. An example:

Alice:  gish put @Alice AA.md aa.py bb.py  # upload a gist with 3 files

Bob:    gish list @Alice        # list Alice's gists
Bob:    gish get @Alice AA.md   # download all 3 files

Here @Alice means "user Alice on gist.github", and AA.md is her gist with 3 files.
The idea is that @Alice files is like ~Alice/files in Unix, just on a remote file server.

gish commands: list get put web

gish list / get / put / web  [@user or @id:]  [--options]  [filenames]

        # list a user's gists on gist.github --
gish list @user
    --days 30  # gists updated in the last 30 days, default 36500

        # download a gist, one or more files --
gish get @user gistfile
    finds the newest gist containing "gistfile", and downloads its files.
    --to directory  # where to put them, default ./tmp/

        # upload local files to gist.github --
gish put @user  [--options]  localfile ...
    either updates a gist that has "localfile", or creates a new one.
    Local "dir/file ..." -> "file ..." without "dir/" on gist.github.
    --auth user,token   # default: $GIST_AUTH
    -d "description"    # default: $HOST, $PWD, date and time
    --private           # default is public

        # open a web browser window on a gist --
gish web @user [gistfile]

If you know the hex id of a gist, say 1234abcdef, use @id:1234abcdef instead of @user; see "Gist ids" below.

@user or @id: are optional. If neither is given, $GIST_USER is looked up in the Unix environment, then if that's not set, $USER . (For how to set these, see export in shell or setenv in csh.)

get @user My.md downloads the newest gist that has a file My.md; similarly, put @user My.md ... either updates the newest gist that has a file My.md, or creates a new gist. See "unique names" below.

put needs write permission for @user or @id on gist.github, either --auth user,token (separated by a ",") or $GIST_AUTH. (A token is a kind of password for an account on github or gist.github; see github access-token .

Try gish list @user for some users you know. The output looks like this:

{ 0-Gradient-descent-with-0-crossing.md  1-gd_cross0.py
description: Gradient descent with 2-point line fit where gradients cross 0
  2590  0-Gradient-descent-with-0-crossing.md
  3637  1-gd_cross0.py
updated: 2017-06-20 12:12:02z
id: 92bc951d542a64a3f164f98bce84dabe
url: https://gist.github.com/92bc951d542a64a3f164f98bce84dabe
}
...

# Some shell one-liners that use `list`:
# a user's gist files and descriptions:
    gish list @user | egrep '^{|^desc'

# gists that have exactly the same files:
    gish list @user | egrep '^{' | sort | uniq -d

Why gish ?

Why a CLI, when one can go to https://gist.github.com/user in a browser, then click "new gist", cut/paste to or from local files, etc. ?

  1. names are easier to remember and pass around than long hex ids
  2. list, get and put should be as easy as ls and cp in Unix
  3. a CLI can cooperate with other tools: shell scripts, grep ...

Of course, many people prefer GUIs to CLIs — depends on the task, and on what you're used to.

Give each gist a unique name

Say a gist has 3 files, e.g. put @Alice AA.md aa.py bb.py. Then it has 3 names, any of which could be used to download all 3 files:

get @Alice AA.md
get @Alice aa.py
get @Alice bb.py

But what if @Alice has several gists with an AA.md or an aa.py ? The newest with an AA.md might be a different gist than the newest one with an aa.py. Or what if there are dozens of gists all with a README.md ? Non-unique names can be confusing ! Give each gist a unique name: Nicename-readme.md, not README.md.

Gist ids

Every gist has a long "id" of hex digits 0-9 a-f, e.g. 1234abcdef, which is unique for all the zillions of gists on gist.github . In gish, a gist may be accessed either by @user or by @id:1234abcdef. (Internally, gish get / put @user filename does gish @user list to build a dictionary of filenames to ids, newest to oldest, then looks for a gist with filename.)

Filenames

If there are several files in a gist, gist.github displays them sorted in ascii order: . 0-9 A-Z _ a-z . Capitalize My.md to put it at the top, before other filenames starting with lower-case a-z. To force a particular order, name the files e.g. 0-My.md 1-my.py 2-test.py ... . (The first file in a gist is usually a .md, github-flavored-markdown, but it can be anything.)

I recommend using only the characters 0-9 A-Z a-z _ - . in filenames, to keep them portable. Blanks in filenames may not work in some shell scripts; use minus signs instead. (I don't think git supports Unicode in filenames, but see how-to-handle-asian-characters-in-file-names-in-git-on-os-x and sys.getfilesystemencoding.)

Etc. etc.

How it works: gish is glue software, with the python modules json and requests doing the work:
        Unix files  ↔  Data  ↔  json  ↔  requests  ↔  gist API
where Data is a Python dict { filename : data ... }, all in memory.

https://api.github.com/gists/<id> is a nice json viewer of the complete gist data structure.

If you get requests - OpenSSL errors on macOS, install python 2.7.15, which has a builtin copy of OpenSSL; see here.

gish license: creativecommons, NonCommercial ShareAlike.

Notes on file trees  ↔  flat gists

Gists are flat, with no directory/file*. How to transfer a file tree  ↔  a flat gist ?

  • gish put .../dir/file* uploads to file* in a gist, dropping any leading .../dir
  • gish get downloads the files in a gist to the --to directory, by default ./tmp/, after which you can move and rename them as you please.

One could map local files doc/My.md src/top.py ...  ↔  flat gist files doc,My.md src,top.py ... but no single way of doing this will satisfy everyone, and it's easily scripted. (Beware: put ../../dir/../file ?) In short, flat  ↔  flat is simple and good enough.

Another way is to pack / unpack dir/file* in one big text file, see man shar.

Bugs, todo s

Print a warning on get non-unique-name.
Another gist name: desc:regex.
Get / put .zip ?

gish was written in 2014, in python2. Thus it gets and puts only text files — afaik, binary  ↔  requests unicode is not possible in python2. (python3 ? raw_url ?)

Links

github gists API
github3py — looks awfully heavy to just list / get / put gists
gistup — uses git, not requests

Comments welcome

I'd appreciate a sharp-eyed doc reader, before releasing the code: doc first.

cheers
— denis-bz-py at t-online dot de

Last change: 2018-07-10 July

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment