Citing BlakeClass https://www.reddit.com/r/AskReddit/comments/24vzgl/you_just_won_a_656_million_dollar_lottery_what_do/chba4bf/
Now you're fucked.
No really.
You are.
Citing BlakeClass https://www.reddit.com/r/AskReddit/comments/24vzgl/you_just_won_a_656_million_dollar_lottery_what_do/chba4bf/
Now you're fucked.
No really.
You are.
A running example of the code from:
This gist creates a working example from blog post, and a alternate example using simple worker pool.
TLDR: if you want simple and controlled concurrency use a worker pool.
# Rate limiting with Celery + Django + Redis | |
# Multiple Fixed Windows Algorithm inspired by Figma https://www.figma.com/blog/an-alternative-approach-to-rate-limiting/ | |
# and Celery's sometimes ambiguous, vague, and one-paragraph documentation | |
# | |
# Celery's Task is subclassed and the is_rate_okay function is added | |
# celery.py or however your App is implemented in Django | |
import os | |
import math |
defmodule AppWeb.Component.FormFromSchema do | |
use AppWeb, :live_component | |
attr :changeset, :any, required: true | |
attr :event_suffix, :string, default: "" | |
attr :schema, :any, required: true | |
attr :uploads, :any, default: nil | |
attr :visibility_rules, :any, default: %{} | |
attr :field_type_rules, :any, default: %{} |
Celery worker blocks on rate limited task | |
========================================= | |
by github.com/miraculixx | |
Problem: | |
If a worker has a rate_limit active on some task, and that task | |
arrives (is received) more often than the rate limit interval, all | |
worker processes will block on these task instances and stop | |
consuming other tasks as soon as the prefetch count has maxed out |
import os | |
import unittest | |
from airflow.models import DagBag | |
class TestDags(unittest.TestCase): | |
""" | |
Generic tests that all DAGs in the repository should be able to pass. | |
""" |
# This is not used unless SQLALCHEMY_BINDS is not present | |
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://{username}:{password}@{hostname}/{database}?charset=utf8' | |
SQLALCHEMY_BINDS = { | |
'master': 'mysql+pymysql://{username}:{password}@{hostname}/{database}?charset=utf8', | |
'slave': 'mysql+pymysql://{username}:{password}@{hostname}/{database}?charset=utf8' | |
} |
library(data.table) | |
prep_data=F | |
if(prep_data){ | |
# https://www.insee.fr/fr/statistiques/2520034 | |
carreaux=foreign::read.dbf("external_data/carroyage_200m.dbf") | |
carreaux=data.table(carreaux) | |
head(carreaux) | |
carreaux[,idINSPIRE:=as.character(idINSPIRE)] |