Skip to content

Instantly share code, notes, and snippets.

@rdmurphy
Last active April 19, 2022 18:07
Show Gist options
  • Star 10 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rdmurphy/d027bbe7bf6f8e6bbb5a0ad3c2194b86 to your computer and use it in GitHub Desktop.
Save rdmurphy/d027bbe7bf6f8e6bbb5a0ad3c2194b86 to your computer and use it in GitHub Desktop.
Ryan's favorite command line tools

My favorite command line tools (2021-09-09)

I wrote this in response to Gurman's tweet. I remember answering in News Nerdery, but it was sadly lost to the void. :(

The CSV Swiss-army knife that trumps all others. Written in Rust, so it's absurdly fast in a way that's still kind of shocking.

VisiData describes itself as a "data multitool" and it's an apt description. VisiData is the command-line spreadsheet you never knew you wanted, and it has basically replaced anything I'd normally use Excel for. You can make quick command line charts and visuals, slice and dice data files from a variety of inputs (I like shoving SQLite DBs in there!). Take a look at this blog post of ten things you can do with it to get a taste.

Big shoutout to Jeremy-Singer Vine for introducing a bunch of us to this tool at NICAR. He also prepared this great intro for VisiData I highly recommend.

You're probably familiar with Mapshaper the site, but there's also an extremely capable command line interface to it as well. Much like VisiData, I'd describe Mapshaper's CLI as a "geodata multitool." It's also possible to import the library in Node.js to chain and build commands programmatically.

Slice and filter JSON very, very quickly. I use this to reformat data feeds on the fly and save myself from writing tedious scripts in Python or Node just to pull out some fields.

Download videos from online services on the command line. Honestly surprised how much I use this. Despite the name it works on much more than just YouTube. Many times there have been reporters who reach out in a scramble trying to backup or grab a video clip before it disappears and youtube-dl has rarely failed me.

Are you working with video and audio files? You probably need ffmpeg. (Or you're already using it! So many things depend on this.)

A tool I've been using more and more lately! Particularly the ability to pipe in JSON or CSV data to create SQLite DBs on the fly.

(It'd be kind of a stretch to call Datasette a command-line tool, but it deserves an honorable mention because it 1) complements sqlite-utils so well and 2) it's very cool. Create a SQLite DB with sqlite-utils, open it in Datasette, and get to exploring.)

Is there anything wrong with curl? No. Is httpie nicer to use in most scenarios? Yes.

Like jq above, but for HTML selection. Written in Go so like xsv it's stupid fast. Save yourself from writing scripts to grab values from a page!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment