Skip to content

Instantly share code, notes, and snippets.

@danieldanciu
Created September 14, 2022 06:35
Show Gist options
  • Save danieldanciu/081f7d9ffdb5b51222b238d5315b744a to your computer and use it in GitHub Desktop.
Save danieldanciu/081f7d9ffdb5b51222b238d5315b744a to your computer and use it in GitHub Desktop.
Code snippet of Cradle's Firestore watcher
def callback(docs: List[firestore.DocumentSnapshot],
changes: List[DocumentChange],
_: DatetimeWithNanoseconds):
try:
for document, change in zip(docs, changes):
if change.type in {ChangeType.ADDED, ChangeType.MODIFIED}:
job_doc = firestore.DocumentReference(*document.reference._path, client=db_client)
if not job_doc.path.startswith(f'{firestore_prefix}users'):
continue
pre_status = document.get('status')
if pre_status in ['PENDING', 'MSA_COMPLETE']:
if pre_status == 'PENDING': # check that quota wasn't exceeded (by tampering with client check)
user_doc = job_doc.parent
active_job_count = mark_stale_jobs(user_doc)
if active_job_count >= MAX_ACTIVE_COUNT:
job_doc.update(
{'status': 'FOLDING_FAILED', 'pdb_error': 'Maximum number of active jobs exceeded'})
logging.warning(f'Job {job_doc.id} is not sent for processing. Too many active jobs')
continue
logging.info(f'Found new job {document.reference.path} with status {pre_status}')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment