Skip to content

Instantly share code, notes, and snippets.

View akaariai's full-sized avatar

Anssi Kääriäinen akaariai

  • Finnish National Institute for Health and Welfare
  • Espoo, Finland
View GitHub Profile
This is a heads-up post about some planned changes to the ORM and specifically to the expressions API. This affects how the following features are dealt with inside the ORM:
- F-expressions (and other ExpressionNodes)
- Aggregates
- Anything using SQLEvaluator (django.db.models.sql.expressions)
While the changes target private APIs, those APIs have remained stable for a long time and I expect there to be users of the above APIs. The main concern is that the planned changes will break existing code. We need feedback from users about how these changes will break existing code. If we find some common cases we might be able to add backwards compatibility code paths for those cases.
There are two main reasons for the change. First, the change allows for a lot of nice new features - doing conditional aggregates, aggregates using expressions and writing custom expressions, all this using public APIs. The second reason is that the current coding is somewhat complex, and that complexity makes it hard to write
What I did in pull/52
- I tried first
- git checkout -b pull_52 upstream/master
- curl https://github.com/django/django/pull/52.patch | git am
but I got merge errors and decided to go another route (git am --abort; git branch -d pull_52)
- added remote (git remote add dstufft https://github.com/dstufft/django.git)
- created a local branch as above
- git merge --squash dstufft ticket/xxxxx
- got conflict. At this point I used git blame to see why there was a conflicting smart_str() call removal, spotted the commit with git blame which said the smart_str() wasn't needed any more. Check!
- committed the work so far.
ForeignKey consists of:
- in the defining model, the model has a descriptor at field.name (that
is, somename in somename = ForeignKey(...)). The descriptor is
responsible for fetching the object when needed, and setting the
field.attname (explained below) to a value when the field.name
is set.
- field.attname has the raw foreign value, usually the "pointed to"
model's pk value. So, when an object is loaded from DB, then the
field.attname of the obj is set to the raw value. When the user
@akaariai
akaariai / gist:5132829
Last active December 14, 2015 18:59
Call-through middleware
A quick suggestion - turn middleware process_request, process_response
and process_exception into single method, process. For example tx
middleware would be then written as:
class TXMiddlware(object):
def process(self, request):
with transaction.atomic():
return request.process()
The request.process() will call next middleware with process() in the mw
This is a random collection of problems with the ORM API regarding multivalued relations.
A multivalued relation is any relation for reverse foreign key queries or relations generated by ManyToManyField.
For multivalued relations an essential concept is that we can query the same relation multiple times inside a single query. For example, one can ask a question "which bands have members who are both older than 50 and taller than 180 cm" (query A), or a question "which bands have a member who is older than 50 and a member who is taller than 180" (query B).
It is notable that negated queries against multivalued relations use subqueries. The reason is that for example members__age__gte=50 matches any band that has at least single member older than 50 while ~Q(members__age__gte=50) removes all bands having at least one member aged older than 50. If we used a join for the exclude query and a band had two members, one aged 30 and one 60, then that band would be returned by the exclude query as the 30 years old m
# Some test classes. We want to match the results of Python
# mro.
class Field(object):
def __eq__(self, other):
print('__eq__ of Field called')
class MyField(Field):
def __eq__(self, other):
print('__eq__ of MyField called')
import itertools
# Some test classes. We want to match the results of Python
# mro.
class Field(object):
def __eq__(self, other):
print('__eq__ of Field called')
class MyField(Field):
@akaariai
akaariai / join_promotion
Last active February 26, 2024 12:51
Django join promotion
=========================
Join promotion in the ORM
=========================
[NOTE: We need better terms than promote and demote for changing the join
type. These terms are extremely easy to mix up. Maybe the ORM methods could
be to_inner_joins and to_louter_joins instead of promote_joins and demote_joins?
I tried to clean up the mis-usages of promotion/demotion but there could still
be some cases where these are mixed up]