Skip to content

Instantly share code, notes, and snippets.

@chathurawidanage
Created June 1, 2021 01:21
Show Gist options
  • Save chathurawidanage/2422c03dc9ad597b8a08d5b02c44faeb to your computer and use it in GitHub Desktop.
Save chathurawidanage/2422c03dc9ad597b8a08d5b02c44faeb to your computer and use it in GitHub Desktop.
from pycylon import DataFrame, read_csv, CylonEnv
from pycylon.net import MPIConfig
env = CylonEnv(config=MPIConfig())
print("Hello from Worker : ", env.rank)
df1 = read_csv("file1_"+str(env.rank)+".csv")
df1.set_index([0])
df2 = read_csv("file2_"+str(env.rank)+".csv")
df2.set_index([0])
join = df1.merge(df2, left_on=[0], right_on=[0], env=env)
print(join)
env.finalize()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment