Skip to content

Instantly share code, notes, and snippets.

@jmquintana79
Last active July 19, 2021 07:57
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save jmquintana79/30462b0f2b60107742c394fa116fbd41 to your computer and use it in GitHub Desktop.
Save jmquintana79/30462b0f2b60107742c394fa116fbd41 to your computer and use it in GitHub Desktop.
Store Pandas dataframe content into MongoDb
from pymongo import MongoClient
from odo import odo
import pandas as pd
# open connection
connection = MongoClient()
# pandas df creation
DF = pd.DataFrame({'A': [1,2,3,4,5,6,7], 'B':[10,20,30,40,50,60,70]})
# database connection
db = connection.<database>
# insert df into mongo <table>
odo(DF, db.<table>)
@arnokurniawan
Copy link

arnokurniawan commented Jan 24, 2020

what is DF in line 13 ?

@jmquintana79
Copy link
Author

DF is a Pandas Dataframe. Yes, I am going to modify my snippet. Thank you.

@jameslin101
Copy link

sorry just found this thread from google search. looks like odo doesn't support the latest pandas? getting
*** ImportError: cannot import name 'tslib' from 'pandas'

@jmquintana79
Copy link
Author

Sorry because answering so late. I didn't know about that. When I created my snippet I tested it with a previous version of Pandas (I cannot remember which one) and it worked. If I am honest, I don't have used odo library yet. Maybe I should check it and/or at least including a list of requeriments with libraries and their respective versions. Thanks

@jameslin101
Copy link

Thanks for the response. I found a solution that worked out for me:

To save:

mongo_dict = json.loads(df.T.to_json())
mongo.db.collection_name.save(mongo_dict)

To load:

mongo_dict = mongo.db.collection_name.find_one({"_id": id})
df= pd.DataFrame.from_dict(mongo_dict, orient='index')

@jmquintana79
Copy link
Author

Thank you very much. Interesting.

@bharath-kumarn
Copy link

data='some example dataframe'
data.reset_index(inplace=True)
data_dict = data.to_dict("records")
# Insert collection
collection.insert_many(data_dict)

@jmquintana79
Copy link
Author

So simple. Thank you very much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment