Skip to content

Instantly share code, notes, and snippets.

@nymous
nymous / README.md
Last active July 4, 2024 07:33
Logging setup for FastAPI, Uvicorn and Structlog (with Datadog integration)

Logging setup for FastAPI

This logging setup configures Structlog to output pretty logs in development, and JSON log lines in production.

Then, you can use Structlog loggers or standard logging loggers, and they both will be processed by the Structlog pipeline (see the hello() endpoint for reference). That way any log generated by your dependencies will also be processed and enriched, even if they know nothing about Structlog!

Requests are assigned a correlation ID with the asgi-correlation-id middleware (either captured from incoming request or generated on the fly). All logs are linked to the correlation ID, and to the Datadog trace/span if instrumented. This data "global to the request" is stored in context vars, and automatically added to all logs produced during the request thanks to Structlog. You can add to these "global local variables" at any point in an endpoint with `structlog.contextvars.bind_contextvars(custom

@ruanbekker
ruanbekker / promtail_docker_logs.md
Last active May 31, 2024 23:52
Docker Container Logging using Promtail
@daimon99
daimon99 / resources.py
Last active February 28, 2020 13:44
Custom django import export resource class, using verbose_name as export file header, and support clean field in the resource class。自定义的 django import export resource 类,支持用 verbose_name 作为导出/导入文件的列名,并支持在 Resouce 类中直接 clean 字段值
# coding: utf-8
from collections import OrderedDict
from copy import deepcopy
from django.core.exceptions import ValidationError
from django.utils.encoding import force_str
from import_export import resources, widgets
from . import models as m
/var/log/traefik.log
{
compress
create 0640 root root
daily
delaycompress
missingok
notifempty
rotate 5
@meitei11
meitei11 / nginx-logs-template.json
Created September 11, 2018 11:48
Sample elasticsearch index template for nginx logs
_template/template_nginx_access_log
{
"index_patterns" : "*-nginx-access*",
"order" : 1,
"settings" : {
"number_of_shards" : 2,
"number_of_replicas" : 0,
"codec" : "best_compression"
},
"mappings" : {
@ngtvspc
ngtvspc / EXIF_remover.py
Created January 9, 2018 23:00
Remove EXIF metadata from images
# Uses the Python Imaging Library
# `pip install Pillow` works too
from PIL import Image
image_filename = "picture_with_EXIF.jpg"
image_file = open('image_filename)
image = Image.open(image_file)
# next 3 lines strip exif
image_data = list(image.getdata())
@Bernix01
Bernix01 / log.py
Created January 3, 2018 16:58
drf LogEntry viewset Mixin
'''
Extracted and modified from django-model-logging
It used it's own LogEntry model but since django
has it's own LogEntry maybe someone would want to
register in the same model instead of creating a
new one.
'''
from django.contrib.admin.models import LogEntry, ADDITION, CHANGE, ContentType, DELETION
from django.utils.translation import gettext as _
@lorne-luo
lorne-luo / python_console_for_pycharm.py
Last active March 17, 2020 10:03
django_extensions' shell_plus for Pycharm's python console
# open Pycharm settings->Build,Execution,Deployment->Console->Django Console
# 1.add env var: DJANGO_SETTINGS_MODULE -> settings.settings
# 2.copy below into Starting script
# open preferences->keymap->search `Python Console`->create new shortcut `cmd + 3`
from pprint import pprint
from django_extensions.management.shells import import_objects
from django.core.management.color import no_style
from datetime import datetime
@victorono
victorono / remove_duplicates.py
Last active April 26, 2024 17:57
Django - remove duplicate objects where there is more than one field to compare
from django.db.models import Count, Max
unique_fields = ['field_1', 'field_2']
duplicates = (
MyModel.objects.values(*unique_fields)
.order_by()
.annotate(max_id=Max('id'), count_id=Count('id'))
.filter(count_id__gt=1)
)
@Skyross
Skyross / task_with_lock.py
Last active February 5, 2024 05:51
Celery Task with lock
from celery import Task
from django.conf import settings
from django.core.cache import caches
from celery.utils.log import get_task_logger
logger = get_task_logger(__name__)
# noinspection PyAbstractClass
class TaskWithLock(Task):
"""