CountTimeoutLimitPaginator - Paginator that enforced a timeout on the count operation.

time_limited_paginator.py

class TimeLimitedPaginator(Paginator):
    """
    Paginator that enforced a timeout on the count operation.
    When the timeout is reached a "fake" large value is returned instead,

    Why does this hack exist? On every admin list view, Django issues a
    COUNT on the full queryset. There is no simple workaround. On big tables,
    this COUNT is extremely slow and makes things unbearable. This solution
    is what we came up with.
    """

    @cached_property
    def count(self):
        # We set the timeout in a db transaction to prevent it from
        # affecting other transactions.
        with transaction.atomic(), connection.cursor() as cursor:
            cursor.execute('SET LOCAL statement_timeout TO 200;')
            try:
                return super().count
            except OperationalError:
                return 9999999999

Hi,

Thanks for this @hakib
Modified it to use with MySql backend

            cursor.execute('SET SESSION MAX_EXECUTION_TIME=2000;')

ARG........
I now realize after many lost hours that the main reason I was repeatedly receiving strange errors that I was not able to interpret
is that
cursor.execute('SET LOCAL statement_timeout TO 200;')

was not recognized by my sqlite server !

I have been searching on internet how to implement a request execution timeout on sqlite, without any success.

Could one of you tell me how I could do that?
Thanks in advance

Hey @formacude,

As far a I can tell, SQLite does not provide a way to limit the time of a single query. the way MySQL and PostgreSQL do. The closest I could find is this.

If you come up with a solution for SQLite please share ;)