Skip to content

Instantly share code, notes, and snippets.

@emesday

emesday/json-array-to-json-object-per-line.sh

Created August 15, 2017 08:51
Show Gist options
  • Save emesday/0cc7bf0072fe8195b041118f3409c7bf to your computer and use it in GitHub Desktop.
Save emesday/0cc7bf0072fe8195b041118f3409c7bf to your computer and use it in GitHub Desktop.
Download ZIP
나무위키:데이터베이스 덤프 변환 - one json object per line
Raw
json-array-to-json-object-per-line.sh
# namuwiki_170327기준 나무위키 덤프는 Json Array로 저장되어 분산 처리를 할 수 없음
# 분산 처리를 위해서 한 라인에 하나의 Json Object로 저장할 필요가 있음
# input: namuwiki_170327.json
# output: output.json
jq -nc --stream 'fromstream(1|truncate_stream(inputs))' namuwiki_170327.json > output.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment