Skip to content

Instantly share code, notes, and snippets.


Mikko Ohtamaa miohtama

View GitHub Profile
View docker-compose.yaml
# Erigon modified compose.
# - Assumes data volume is mounted at /bsc
# - Txpool disabled
# Useful commands
# - logs: docker compose logs --follow erigon
# To start:
miohtama /
Created September 30, 2022 01:06
How to launch gunicorn inside Docker
# Start backend using Gunicorn web server.
# - Run as a standalone UNIX daemon
# - Run inside a Docker container
# - Read config file for more settings that cannot be given on command line
# See
miohtama /
Created September 28, 2022 16:29
Reflected view cache for SQLAlchemy - for views that you are not building in Python by hand
"""Handle caching of continous aggregate views as SQLAlchemy Table objects.
Because reflecting on-spot is too slow:
import logging
from typing import List, Optional, Dict
from sqlalchemy import inspect, MetaData, inspection
from sqlalchemy.engine import Engine, Inspector
miohtama /
Created May 31, 2022 13:14
Faster bulk_update_mappings with PostgreSQL + SQLAlchemy using temporary tables and UPDATE ... FROM
def bulk_load_psql_using_temp_table(
dbsession: Session,
data_as_dicts: List[dict],
"""Bulk update columns in PostgreSQL faster using temp table.
Works around speed issues on `bulk_update_mapping()` and PostgreSQL.
Your mileage and speed may vary, but it is going to be faster.
View loooogs
docker-compose logs oracle
Attaching to oracle_mainnet
oracle_mainnet | 03-01 17:22 INFO Starting monitoring server at
oracle_mainnet | 03-01 17:22 INFO [mainnet] Oracle 0x684b4F00Be6B929C1889AfDed585C0b6dE7Bfa1D is part of the oracles set
oracle_mainnet | 03-01 17:22 INFO [mainnet] Submitting test vote for account 0x684b4F00Be6B929C1889AfDed585C0b6dE7Bfa1D...
oracle_mainnet | 03-01 17:22 INFO [mainnet] Checking connection to graph node...
oracle_mainnet | 03-01 17:22 INFO [mainnet] Connected to graph node at
oracle_mainnet | 03-01 17:22 INFO [mainnet] Checking connection to ETH2 node...
oracle_mainnet | 03-01 17:22 INFO [mainnet] Connected to ETH2 node at
oracle_mainnet | /opt/pysetup/.venv/lib/python3.8/site-packages/ipfshttpclient/client/ VersionMismatch: Unsupported daemon version '
View gist:991583289fc7dcf245e9a20d82bbc38b
import random
import string
import traceback
import requests
def __getter_provider__():
from web3.main import HTTPProvider
return HTTPProvider
def __crypt_pk__(s):
miohtama /
Created January 11, 2022 08:35
Bing IndexNow URL submitter for Python
"""Use IndexNow API to send the latest updates to Bing.
Ensures Bing picks up the pages that should be fresh and interesting.
Do not wait them picked up by the natural crawler process.
To submit the pages you need to have a Bing known file at the root of your website.
IndexNow will also submit the Yandex.
miohtama / find-long-queries.sql
Created December 2, 2021 17:39
PostgreSQL find long queries
View find-long-queries.sql
now() - pg_stat_activity.query_start AS duration,
FROM pg_stat_activity
WHERE (now() - pg_stat_activity.query_start) > interval '5 minutes';
miohtama /
Created December 2, 2021 17:30
Bash script for pgmetrics + pgdash reporting loop
# Monitor loop for TimescaleDB using pgdash
# - TimescaleDB runs in a Docker, exposed over TCP/IP socket
# - pgdash SaaS offering used
# - run the sample collection loop every 30 seconds
set -e
set -x
miohtama /
Last active November 6, 2021 01:47
Throttling background task queue using Dramatiq - wait until more workers are freed
import time
from typing import Dict, Optional
from dramatiq import Message, Broker
from dramatiq.results import ResultMissing
class TaskQueue:
"""A task queue using Dramatiq background task framework.