Skip to content

Instantly share code, notes, and snippets.

View supreme-core's full-sized avatar
:octocat:

I am me supreme-core

:octocat:
View GitHub Profile
@supreme-core
supreme-core / alternative-scheduler-for-celery.py
Created March 7, 2022 05:49 — forked from amitripshtos/alternative-scheduler-for-celery.py
Alternative to celery beat for people who got burned hard
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.executors.pool import ThreadPoolExecutor
from apscheduler.jobstores.memory import MemoryJobStore
from apscheduler.job import Job
import json
import logging
from apscheduler.triggers.cron import CronTrigger
import time
from celery import Celery
from typing import List
@supreme-core
supreme-core / task_with_lock.py
Created March 7, 2022 05:41 — forked from Skyross/task_with_lock.py
Celery Task with lock
from celery import Task
from django.conf import settings
from django.core.cache import caches
from celery.utils.log import get_task_logger
logger = get_task_logger(__name__)
# noinspection PyAbstractClass
class TaskWithLock(Task):
"""
@supreme-core
supreme-core / flask_celery.py
Created March 7, 2022 05:40 — forked from danoctua/flask_celery.py
Sample Flask Celery integration
import flask
from celery import Celery
class FlaskCelery(Celery):
app = None
def __init__(self, *args, **kwargs):
@supreme-core
supreme-core / __init__.py
Created March 7, 2022 05:38 — forked from SergSm/__init__.py
Django + Gunicorn + Celery + Supervisor
from .celery import app as celery_app
__all__ = ('celery_app', )
@supreme-core
supreme-core / Dockerfile
Created March 7, 2022 05:37 — forked from mmautner/Dockerfile
celery docker-compose example
FROM python:3.4
ADD . /app/
WORKDIR /app/
RUN pip install -r requirements.txt
CMD ["echo", "hello"]
@supreme-core
supreme-core / show_long_running_celery_tasks.py
Created March 7, 2022 05:29 — forked from kgantsov/show_long_running_celery_tasks.py
Python function that prints currently running celery tasks that are running longer that number of seconds specified in threshold param
from datetime import datetime
import time
import kombu.five
from celery.task.control import inspect
def get_stuck_celery_tasks(threshold=600):
i = inspect()
for worker, tasks in i.active().items():
@supreme-core
supreme-core / revoke_celery_tasks.py
Created March 7, 2022 05:29 — forked from kgantsov/revoke_celery_tasks.py
Helper functions that revokes celery tasks by name and ID
from celery.task.control import revoke
from celery.task.control import inspect
def revoke_tasks_by_name(task_name, worker_prefix=''):
"""
Revoke all tasks by the name of the celery task
:param task_name: Name of the celery task
:param worker_prefix: Prefix for the worker
@supreme-core
supreme-core / README.md
Created March 7, 2022 05:24 — forked from lorne-luo/README.md
Celery 4.1 Intergration with Django 1.11

Celery

Monitor celery by using celery -A freightquotes events

Start celery with beats celery -A freightquotes worker -B -E -l INFO --autoscale=2,1

Note:- 
    -B  beats - to execute schedule tasks
    -l  log-level (INFO/DEBUG)

-E --task-events

@supreme-core
supreme-core / celeryconfig.py
Created March 7, 2022 02:24 — forked from channeng/celeryconfig.py
Celery Flask setup - celeryconfig and celerybeat schedule
from celery.schedules import crontab
CELERY_IMPORTS = ('app.tasks.test')
CELERY_TASK_RESULT_EXPIRES = 30
CELERY_TIMEZONE = 'UTC'
CELERY_ACCEPT_CONTENT = ['json', 'msgpack', 'yaml']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
@supreme-core
supreme-core / bash.py
Created March 7, 2022 00:54 — forked from judotens/bash.py
Celery Run Bash Project
from __future__ import absolute_import
import os
from subprocess import Popen, PIPE
import datetime
import time
from celery import Celery
from celery import states
from os.path import dirname, join