Skip to content

Instantly share code, notes, and snippets.

@napsternxg
Created April 29, 2020 13:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save napsternxg/7b5f1cd4af3178f235008e24ae6ec748 to your computer and use it in GitHub Desktop.
Save napsternxg/7b5f1cd4af3178f235008e24ae6ec748 to your computer and use it in GitHub Desktop.
Print pyspark schema in indented format
def print_schema(schema, level=0):
for item in schema:
elementType = None
elementTypeName = ""
if hasattr(item.dataType, "elementType"):
elementType = item.dataType.elementType
elementTypeName = item.dataType.elementType.typeName()
#print("{}{}".format("\t"*level, item))
print("{}{}\t{}\t{}\t{}".format(
"\t"*level,
item.name,
item.dataType.typeName(),
elementTypeName,
elementType
))
if elementType is not None:
if item.dataType.elementType.typeName() == "struct":
print_schema(item.dataType.elementType.fields, level=level+1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment