Skip to content

Instantly share code, notes, and snippets.

@chathurawidanage

chathurawidanage/cylon_dst.py

Last active June 1, 2021 01:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chathurawidanage/71f48c60ae49964c8005dedf90d601be to your computer and use it in GitHub Desktop.
Save chathurawidanage/71f48c60ae49964c8005dedf90d601be to your computer and use it in GitHub Desktop.
Download ZIP
Raw
cylon_dst.py
from pycylon import DataFrame, read_csv, CylonEnv
from pycylon.net import MPIConfig
env = CylonEnv(config=MPIConfig())
print("Hello from Worker : ", env.rank)
df1 = read_csv("file1.csv")
df1.set_index([0])
df2 = read_csv("file2.csv")
df2.set_index([0])
join = df1.merge(df2, left_on=[0], right_on=[0], env=env)
print(join)
env.finalize()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment