Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
# import sql function pyspark
import pyspark.sql.functions as f
# null values in each column
data_agg = my_data.agg(*[f.count(f.when(f.isnull(c), c)).alias(c) for c in my_data.columns])
data_agg.show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.