Skip to content

Instantly share code, notes, and snippets.

@bartek
Created December 30, 2011 17:53
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bartek/1540795 to your computer and use it in GitHub Desktop.
Save bartek/1540795 to your computer and use it in GitHub Desktop.
efficient bulk_delete for Django
"""
This is a quick hack that adds a bulk_delete function, which given a queryset will run the Django
implemented DeleteQuery. The purpose of this is when you have thousands (or in my case, millions)
of records to delete, I did not want to get the pk_list of every record and store that into memory,
like Django does naturally, and instead just execute a single query with its WHERE clauses.
There are probably some issues with this, in regards to ForeignKeys and cascading. More info can
be found in this ticket, which looks to implement a bulk_delete functionality into the core:
https://code.djangoproject.com/ticket/9519
You can probably throw this code into a Manager object and then it's simply to use it from a model class, e.g.:
MyModel.my_manager.bulk_delete(MyModel.objects.filter(expire_date__lte=datetime.now())
"""
from django.db.models.sql.subqueries import DeleteQuery
def bulk_delete(qs):
"""
Single argument, a queryset.
WARNING: This should not be used with models that have ForeignKey
relationships as they will most likely not do the expected cascading
and other post-delete signals.
"""
delete_query = qs.query.clone(BulkDeleteQuery)
delete_query.delete_batch(using=qs.db)
class BulkDeleteQuery(DeleteQuery):
"""
Override the existing DeleteQuery.delete_batch method to simply take the
existing where clause and execute that, rather than sorting out a primary
key list. This relieves the host from having to store every single instance
into memory, and instead just executes the sql code with the where clause(s).
"""
def delete_batch(self, using, field=None):
if not field:
field = self.model._meta.pk
self.do_query(self.model._meta.db_table, self.where, using=using)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment