Skip to content

Instantly share code, notes, and snippets.

@kovid-r

kovid-r/pyspark_cheatsheet_remove_columns.py

Created June 11, 2020 10:27
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kovid-r/69766cfc6d8c69a9a6500e4b960044bc to your computer and use it in GitHub Desktop.
Save kovid-r/69766cfc6d8c69a9a6500e4b960044bc to your computer and use it in GitHub Desktop.
Download ZIP
Drop columns from a DataFrame PySpark Cheatsheet
Raw
pyspark_cheatsheet_remove_columns.py
# Remove a column from a DataFrame
df.drop('this_column')
# Remove multiple columns in a go
drop_columns = ['this_column', 'that_column']
df.select([col for col in df.columns if column not in drop_columns])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment