Skip to content

Instantly share code, notes, and snippets.

@canimus
Created March 20, 2020 18:55
Show Gist options
  • Save canimus/824b7e3ed11655373a34e8f853c5cfbb to your computer and use it in GitHub Desktop.
Save canimus/824b7e3ed11655373a34e8f853c5cfbb to your computer and use it in GitHub Desktop.
Spark custom separator reader
sc.binaryFiles(new_file).values().flatMap(lambda x: x.decode("iso-8859-1").split(chr(172))).map(lambda x: x.split(chr(171)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment