Skip to content

Instantly share code, notes, and snippets.

@nicksieger
Last active April 18, 2019 21:07
Show Gist options
  • Save nicksieger/704c5a98646c5e473800ae721e7f4748 to your computer and use it in GitHub Desktop.
Save nicksieger/704c5a98646c5e473800ae721e7f4748 to your computer and use it in GitHub Desktop.

JQ Commands

Here are some handy jq scripts that I've found useful.

With AWS CLI

Since the AWS CLI outputs JSON this is a natural fit. Sometimes the built-in --query doesn't quite go far enough.

Examples:

aws lambda list-event-source-mappings | jq -c '.EventSourceMappings[] | { uuid: .UUID, arn: .EventSourceArn, state: .State }'

Working with Elasticsearch output

This command transforms a search result blob into a list of JSON objects suitable for input into ES's "bulk" command.

curl localhost:9200/_search?size=200 | \
  jq -c '.hits.hits[] | { index: { _id: ._id, _index: ._index, _type: ._type } },._source' \
  > bulk.txt

So, for a simple, no-frills replication strategy from localhost:9200 to localhost:9201, consider this pipeline:

curl localhost:9200/_search?size=200 | \
  jq -c '.hits.hits[] | { index: { _id: ._id, _index: ._index, _type: ._type } },._source' | \
  curl -s -H "Content-Type: application/x-ndjson" -X POST localhost:9201/_bulk --data-binary @-; echo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment