Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
# import sql function pyspark
import pyspark.sql.functions as f
# null values in each column
data_agg = my_data.agg(*[f.count(f.when(f.isnull(c), c)).alias(c) for c in my_data.columns])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment