Skip to content

Instantly share code, notes, and snippets.

@EvaSDK
Last active August 29, 2015 14:25
Show Gist options
  • Save EvaSDK/1f1f097300b691d80269 to your computer and use it in GitHub Desktop.
Save EvaSDK/1f1f097300b691d80269 to your computer and use it in GitHub Desktop.
SQL:
WITH mt_to_clean AS (
SELECT mt.key
FROM
master_table AS mt
LEFT JOIN secondary_table_1 AS position
ON mt.key = position.mt_key
LEFT JOIN secondary_table_2 AS bvd_copy
ON mt.key = bvd_copy.mt_key
WHERE
position.mt_key IS NULL
AND
bvd_copy.mt_key IS NULL
)
DELETE FROM mt
WHERE mt.key = mt_to_clean.key
OR
DELETE FROM mt
WHERE mt.key IN (
SELECT mt.key
FROM
master_table AS mt
LEFT JOIN secondary_table_1 AS position
ON mt.key = position.mt_key
LEFT JOIN secondary_table_2 AS bvd_copy
ON mt.key = bvd_copy.mt_key
WHERE
position.mt_key IS NULL
AND
bvd_copy.mt_key IS NULL
)
----------------------
SQLAlchemy:
unreachable_mt_query = session.query(MT.key.label('mt_key')) \
.outerjoin(SecondaryTable1) \
.outerjoin(SecondaryTable2) \
.filter(
SecondaryTable1.mt_key.is_(None),
SecondaryTable2.mt_key.is_(None),
)
unreachable_mt_alias = sa.orm.aliased(unreachable_mt_query.cte(),
name='unreachable_mt')
session.query(MT) \
.join(unreachable_mt_alias,
unreachable_mt_alias.c.mt_key == MT.key) \
.delete(synchronize_session=False)
OR
session.query(MT) \
.outerjoin(SecondaryTable1) \
.outerjoin(SecondaryTable2) \
.filter(
SecondaryTable1.mt_key.is_(None),
SecondaryTable2.mt_key.is_(None),
) \
.delete(synchronize_session=False)
both generate this exception:
Traceback (most recent call last):
File "/Users/gilles/devel/bla/venv/bin/purge-entries", line 9, in <module>
load_entry_point('bla==0.6.3', 'console_scripts', 'purge-entries')()
File "/Users/gilles/devel/bla/bla/tasks.py", line 165, in main
purge_db(int(args.days))
File "/Users/gilles/devel/bla/bla/tasks.py", line 68, in purge_db
for obj in purge_spurious_entries(session):
File "/Users/gilles/devel/bla/bla/purge_entries.py", line 257, in purge_spurious_entries
.delete(synchronize_session=False)
File "/Users/gilles/devel/bla/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2827, in delete
self, synchronize_session)
File "/Users/gilles/devel/bla/venv/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1275, in factory
}, synchronize_session, query)
File "/Users/gilles/devel/bla/venv/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1101, in _factory
return klass(*arg)
File "/Users/gilles/devel/bla/venv/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1267, in __init__
super(BulkDelete, self).__init__(query)
File "/Users/gilles/devel/bla/venv/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1066, in __init__
self._validate_query_state()
File "/Users/gilles/devel/bla/venv/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1084, in _validate_query_state
(methname, )
sqlalchemy.exc.InvalidRequestError: Can't call Query.update() or Query.delete() when join(), outerjoin(), select_from(), or from_self() has been called
@pavel-kolla-kampiki
Copy link

if this gives you all results you want to be deleted:
session.query(MT.id).outerjoin(ST1).outerjoin(ST2).filter(ST1.mt_key.is_(None),ST2.mt_key.is_(None)).all()

this is supposed to delete them:
session.query(MT).outerjoin(ST1).outerjoin(ST2).filter(ST1.mt_key.is_(None),ST2.mt_key.is_(None)).delete(synchronize_session=False)
use DB level cascades if need to propagate deletes

@pavel-kolla-kampiki
Copy link

apparently in 1.0 it's not supposed to work because of this

try:

q = session.query(MT.id).outerjoin(ST1).outerjoin(ST2).filter(ST1.mt_key.is_(None),ST2.mt_key.is_(None)).subquery()
session.query(MT).filter(MT.id.in_(q)).delete()

@EvaSDK
Copy link
Author

EvaSDK commented Jul 22, 2015

Well it did not work previously as it indeed failed to construct the query discarding the bits I was interested in.
The other form I expected to work:

session.query(MT).filter(MT.key.in_(unreachable_mt_query.subquery()))

Did not work with 0.9.8 as well because the table is a polymorphic type and the subquery returned extra columns. This is now fixed in 1.0 it seems. Thank you so much for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment