pymongo vs Motor

pymongo-to-motor.md

pymongo vs Motor

Motor is an async Python driver for MongoDB.

When should I use Motor?

You should use Motor when you're trying to interact with a MongoDB database in an asynchronous context. When you're making something that needs to be asynchronous (like a web server, or most commonly from what I've seen here, Discord bots), you also want all the database calls to be done asynchronously. But pymongo is synchronous, i.e it is blocking, and will block the execution of your asynchronous program for the time that it is talking to the database.

Okay, How do I switch now?!

Thankfully for us, switching from pymongo to Motor isn't too hard, and won't need you to change much code. This process can be roughly summarized as:

Step 1: Install Motor, and import it

Installing can be done with pip - pip install motor After this it can be imported into your file as

import motor.motor_asyncio as motor

(NOTE: The as motor part is just to avoid having to type motor.motor_asyncio everytime)

Step 2: Make a Client and get a database

You must be familiar with pymongo's MongoClient - you will switch that out with motor's equivalent class

client = motor.AsyncIOMotorClient(<connection details>)

Getting a database and a collection is similar to pymongo, (you can do client["database name"] to get the database and you'd do database["collection name"] to get the collection. Or you can also use dot-notation)

Step 3: Make the queries asynchronous

Now add an await before every query you have (except .find(), this will be explained later)

# before
something = collection.find_one({"foo": "bar"})
collection.delete_one({"foobar": 1})
collection.update_many({"foo": "bar"}, {"$set": {"foo": "foobar"}})

# after
something = await collection.find_one({"foo": "bar"})
await collection.delete_one({"foobar": 1})
await collection.update_many({"foo": "bar"}, {"$set": {"foo": "foobar"}})

And you're done! Your queries are no longer blocking the event loop :D

Sidenote; Why aren't .finds awaited?

collection.find only returns a cursor object. No real I/O operation takes place until you actually try accessing the data from the cursor. You can asynchronously loop over the contents of the cursor (with an async for loop), or use the cursor's to_list method to immedietely get all the data (now this has to be awaited!) Meaning,

data = collection.find()    # correct
data = await collection.find()    # wrong!

# retrieving data
# option 1:
async for i in data:
  # do stuff 

#option 2
as_list = await data.to_list(length=None)

to_list takes a length kwarg, which specifies how many documents to retrieve - passing None means you retrieve all documents.

Documentation

another one, when you mentioned that .find() doesn't not require await, should it has to be inside async function to achieve asynchronous?

thank you. I'll save this for future.

what about performance? is there are comparison?

recently I just use FastAPI, should I HAVE to switch to motoer from pymongo?

@chawza
I haven't done any performance benchmarks as such, nor have I seen any online.
But it would probably be better to be using Motor with FastAPI seeing how FastAPI is async, and pymongo would be blocking the event loop in that case (it'd be barely noticeable when you run it locally but it'd still be happening)

another one, when you mentioned that .find() doesn't not require await, should it has to be inside async function to achieve asynchronous?

You can do a .find inside a non-async function but as I've written there - find doesn't do any I/O operations; it just returns a Cursor object, and the I/O operations only take place once you try accessing the data from the Cursor. And this has to be done in an async function.

if fastapi use not async def, it will auto use run_in_executor execute this function ,
motor internal also uses run_in_executor to execute the query, I doubt its performance, because it is not really asynchronous, but base on multi-threaded

There is a true asynchronous MongoDB driver for Python and asyncio: Mongojet. According to its developer, it is 4 times faster than Motor in high concurrency scenarios.

I was wondoring for why 2 libraries for mongodb in python, also after studing asynchronous behaviour, I came across this gist, Thanks really helpful