Skip to content

Instantly share code, notes, and snippets.

@dautovri

dautovri/missOrNull.py

Created January 16, 2018 05:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dautovri/872a9449df359ebff427c05ca5087af9 to your computer and use it in GitHub Desktop.
Save dautovri/872a9449df359ebff427c05ca5087af9 to your computer and use it in GitHub Desktop.
Download ZIP
pyspark show all missing or null values in all columns of dataframe
Raw
missOrNull.py
df.select([count(when(isnan(c) | col(c).isNull(), c)).alias(c) for c in df.columns]).show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment