Skip to content

Instantly share code, notes, and snippets.

Forked from safar/
Last active November 10, 2023 18:25
  • Star 5 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Large Table Paginator for Django: Scale Django admin pages and avoid timeouts.
from django.contrib.admin import ModelAdmin
from .paginator import LargeTablePaginator
class MyTableAdmin(ModelAdmin):
paginator = LargeTablePaginator
show_full_result_count = False # Recommended to avoid another count()
from django.core.paginator import Paginator
from django.db import connection, transaction, OperationalError
from django.utils.functional import cached_property
class LargeTablePaginator(Paginator):
Combination of ideas from:
Overrides the count method of QuerySet objects to avoid timeouts.
- Try to get the real count limiting the queryset execution time to 150 ms.
- If count takes longer than 150 ms the database kills the query and raises OperationError. In that case,
get an estimate instead of actual count when not filtered (this estimate can be stale and hence not fit for
situations where the count of objects actually matter).
- If any other exception occured fall back to default behaviour.
def count(self):
Returns an estimated number of objects, across all pages.
with transaction.atomic(), connection.cursor() as cursor:
# Limit to 150 ms
cursor.execute('SET LOCAL statement_timeout TO 5;')
return super().count
except OperationalError:
if not self.object_list.query.where:
with transaction.atomic(), connection.cursor() as cursor:
# Obtain estimated values (only valid with PostgreSQL)
"SELECT reltuples FROM pg_class WHERE relname = %s",
estimate = int(cursor.fetchone()[0])
return estimate
except Exception:
# If any other exception occurred fall back to default behaviour
return super().count
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment