Skip to content

Instantly share code, notes, and snippets.

@lakshay-arora

lakshay-arora/ohe_pyspark.py

Created November 3, 2019 19:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lakshay-arora/38e12652deb5d98c371ce127ae720968 to your computer and use it in GitHub Desktop.
Save lakshay-arora/38e12652deb5d98c371ce127ae720968 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
ohe_pyspark.py
# create object and specify input and output column
OHE = OneHotEncoderEstimator(inputCols=['Batsman_Index', 'Bowler_Index'],outputCols=['Batsman_OHE', 'Bowler_OHE'])
# transform the data
my_data = OHE.fit(my_data).transform(my_data)
# view and transform the data
my_data.select('Batsman_Name', 'Batsman_Index', 'Batsman_OHE', 'Bowler_Name', 'Bowler_Index', 'Bowler_OHE').show(10)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment