Skip to content

Instantly share code, notes, and snippets.

@chathurawidanage

chathurawidanage/cylon_rank.py

Created June 1, 2021 01:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chathurawidanage/2422c03dc9ad597b8a08d5b02c44faeb to your computer and use it in GitHub Desktop.
Save chathurawidanage/2422c03dc9ad597b8a08d5b02c44faeb to your computer and use it in GitHub Desktop.
Download ZIP
Raw
cylon_rank.py
from pycylon import DataFrame, read_csv, CylonEnv
from pycylon.net import MPIConfig
env = CylonEnv(config=MPIConfig())
print("Hello from Worker : ", env.rank)
df1 = read_csv("file1_"+str(env.rank)+".csv")
df1.set_index([0])
df2 = read_csv("file2_"+str(env.rank)+".csv")
df2.set_index([0])
join = df1.merge(df2, left_on=[0], right_on=[0], env=env)
print(join)
env.finalize()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment