Skip to content

Instantly share code, notes, and snippets.

@lakshay-arora

lakshay-arora/string_index.py

Created November 3, 2019 19:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lakshay-arora/9768022ce05999c9670959d2dc9f1005 to your computer and use it in GitHub Desktop.
Save lakshay-arora/9768022ce05999c9670959d2dc9f1005 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
string_index.py
from pyspark.ml.feature import StringIndexer, OneHotEncoderEstimator
# create object of StringIndexer class and specify input and output column
SI_batsman = StringIndexer(inputCol='Batsman_Name',outputCol='Batsman_Index')
SI_bowler = StringIndexer(inputCol='Bowler_Name',outputCol='Bowler_Index')
# transform the data
my_data = SI_batsman.fit(my_data).transform(my_data)
my_data = SI_bowler.fit(my_data).transform(my_data)
# view the transformed data
my_data.select('Batsman_Name', 'Batsman_Index', 'Bowler_Name', 'Bowler_Index').show(10)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment