Skip to content

Instantly share code, notes, and snippets.

@czzarr
Created March 6, 2019 14:27
Show Gist options
  • Save czzarr/6a55d952fc625e747b038e1090ee24d8 to your computer and use it in GitHub Desktop.
Save czzarr/6a55d952fc625e747b038e1090ee24d8 to your computer and use it in GitHub Desktop.
extract the "users" branch from the database via streaming
jq -nc --stream '
def atomize(s):
fromstream(foreach s as $in ( {previous:null, emit: null};
if ($in | length == 2) and ($in|.[0][0]) != .previous and .previous != null
then {emit: [[.previous]], previous: $in|.[0][0]}
else { previous: ($in|.[0][0]), emit: null}
end;
(.emit // empty), $in) ) ;
atomize(1|truncate_stream(inputs | select(.[0][0] == "users")))' fb_yamm_data.json > yamm_users.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment