Store CSV data into mongodb using python pandas

import_csv_to_mongo

#!/usr/bin/env python
import sys
import pandas as pd
import pymongo
import json



def import_content(filepath):
    mng_client = pymongo.MongoClient('localhost', 27017)
    mng_db = mng_client['mongodb_name'] // Replace mongo db name
    collection_name = 'collection_name' // Replace mongo db collection name
    db_cm = mng_db[collection_name]
    cdir = os.path.dirname(__file__)
    file_res = os.path.join(cdir, filepath)

    data = pd.read_csv(file_res)
    data_json = json.loads(data.to_json(orient='records'))
    db_cm.remove()
    db_cm.insert(data_json)

if __name__ == "__main__":
  filepath = '/path/to/csv/path'  // pass csv file path
  import_content(filepath)

How to read excel sheet using python and i need to store excel values in MongoDB in the form of web page. Would you please give an Idea How to do that ? i am new to python so.

Thanks for sharing

Just an FYI. This solution converts csv values like "0029" to 29.0 float and "" into null. In case anyone is interested to force it all to string, replace line 17 with this:
data = pd.read_csv(file_res,dtype=str,na_filter=False)

Thank you! This helped tons!

Thanks for sharing 🎉 😃

Thank you, this was very helpful!

I need to store values of each column in a different variable (parsing). So that modifications on particular column values are possible before loading to the DB. How can this be done?

thank you for this example, was killing myself to do this seemingly basic thing

@akamath03 did you ever get the solution to your question? 'twould help me out a lot right now.

I am getting an error NameError: name 'file' is not defined - what should I be defining here?

need of a code
where one excel file is created which is uploaded as follow:

result should be:

generate the code for this in python
please help friends

awesome, thanks

Thanks for your help !

why is this happening Undefined variable 'os' ?

why is this happening Undefined variable 'os' ?

Don't forget to import the os module at the top of your script:
import os

why is this happening Undefined variable 'os' ?

Don't forget to import the os module at the top of your script:
import os

yes, got it
Thank You so much

i need to map a csv which has an id field already to be imported to mongo with the _id as this id field.How to do that?
(rhis csv file has an id field)
try:
client = MongoClient(mongouri)
database = client['automation']
database.create_collection('testTableMongo'+str(suffix))
coll = database['testTableMongo'+str(suffix)]
data = pd.read_csv(os.path.abspath('../data/' + file))
payload = json.loads(data.to_json(orient='records'))
coll.remove()
coll.insert(payload)
except OperationFailure as msg:
print("Command skipped: %s", msg)

I'm with the error "name 'file' is not defined"

I'm with the error "name 'file' is not defined"

Hey, kindly post a snippet of your code for easier help

when you use file in python interactive mode, this error occur.
So you should write this line os.path.dirname(__file__) in file.py.

for more reference consider this : https://stackoverflow.com/questions/16771894/python-nameerror-global-name-file-is-not-defined

I need help to load data into mongo database with collection creation from worksheet which contain multiple excel sheets

I need help to load data into mongo database with collection creation from worksheet which contain multiple excel sheets

you can read a sheet with pandas with the sheet_name att.
df = pd.read_excel('my_excel.xlsx', sheet_name='Sheet 1')