Skip to content

Instantly share code, notes, and snippets.

@kovid-r

kovid-r/pyspark_cheatsheet_between.py

Last active October 11, 2022 04:49
Show Gist options
  • Save kovid-r/a836af88fcdc303cd5a93f0371164a58 to your computer and use it in GitHub Desktop.
Save kovid-r/a836af88fcdc303cd5a93f0371164a58 to your computer and use it in GitHub Desktop.
Download ZIP
Filter Between PySpark Cheatsheet
Raw
pyspark_cheatsheet_between.py
# Filter movies with avg_ratings > 7.5 and < 8.2
df.filter((F.col('avg_ratings') > 7.5) & (F.col('avg_ratings') < 8.2)).show()
# Another way to do this
df.filter(df.avg_ratings.between(7.5,8.2)).show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment