Skip to content

Instantly share code, notes, and snippets.

@akoumjian

akoumjian/ndjson.md

Last active July 17, 2019 16:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save akoumjian/887aec2ebb29bdb2074d5319cd94577a to your computer and use it in GitHub Desktop.
Save akoumjian/887aec2ebb29bdb2074d5319cd94577a to your computer and use it in GitHub Desktop.
Download ZIP
Newline delimited JSON Classifier for AWS Glue
Raw
ndjson.md

AWS Glue surpriginsly does not support reading NDJSON out of the box. You have to create a custom classifier, but it is incredibly simple.

Use the pattern $[*] with a custom JSON classifier and it should read the NDJSON schema more or less correctly.

NOTE: Editing classifiers doesn't seem to work well, or they are denormalized in the crawler. Instead, delete and create new classifiers when making adjustments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment