Skip to content

Instantly share code, notes, and snippets.

View miohtama's full-sized avatar
🏠
https://tradingstrategy.ai

Mikko Ohtamaa miohtama

🏠
https://tradingstrategy.ai
View GitHub Profile
@miohtama
miohtama / fast-web3-py-event-decoder.py
Created August 19, 2021 08:08
Fast event fetcher and decoder for web3.py
"""Below is an example for faster JSON-RPC event fetcher.
It skips a lot of steps, like converting raw binary values to corresponding numbers (float, ints),
looking up ABI labels and building `AttributedDict` object.
The resulting Event dictitionary is generated faster, but harder to use and you need to know what you are doing.
"""
def _fetch_events_for_all_contracts(
@miohtama
miohtama / gunicorn-docker-start.sh
Created September 30, 2022 01:06
How to launch gunicorn inside Docker
#!/bin/bash
#
# Start backend using Gunicorn web server.
#
# - Run as a standalone UNIX daemon
# - Run inside a Docker container
# - Read config file for more settings that cannot be given on command line
#
# See https://docs.gunicorn.org/en/stable/settings.html
#
@miohtama
miohtama / reflected-view-cache.py
Created September 28, 2022 16:29
Reflected view cache for SQLAlchemy - for views that you are not building in Python by hand
"""Handle caching of continous aggregate views as SQLAlchemy Table objects.
Because reflecting on-spot is too slow: https://github.com/tradingstrategy-ai/backend/pull/63
"""
import logging
from typing import List, Optional, Dict
from sqlalchemy import inspect, MetaData, inspection
from sqlalchemy.engine import Engine, Inspector
@miohtama
miohtama / sanitycheck.py
Created June 4, 2015 21:22
Perform database sync sanity check to SQLAlchemy models on application startup
import logging
from sqlalchemy import inspect
from sqlalchemy.ext.declarative.clsregistry import _ModuleMarker
from sqlalchemy.orm import RelationshipProperty
logger = logging.getLogger(__name__)
def is_sane_database(Base, session):
@miohtama
miohtama / sheet.py
Last active July 29, 2022 11:47
Creating and sharing Google Sheet spreadsheets using Python
"""Google spreadsheet related.
Packages required: gspread, cryptography, oauth2client, google-api-python-client
For OSX see
* http://stackoverflow.com/a/33508676/315168
"""
import logging
@miohtama
miohtama / smtp-test.py
Created April 19, 2016 16:43
Testing SMTP server from command line using python
# Import smtplib for the actual sending function
import smtplib
# Import the email modules we'll need
from email.mime.text import MIMEText
# Open a plain text file for reading. For this example, assume that
# the text file contains only ASCII characters.
msg = MIMEText("Test")
fp.close()
@miohtama
miohtama / gist:5389146
Created April 15, 2013 15:56
Decoding emails in Python e.g. for GMail and imapclient lib
import email
def get_decoded_email_body(message_body):
""" Decode email body.
Detect character set if the header is not set.
We try to get text/plain, but if there is not one then fallback to text/html.
:param message_body: Raw 7-bit message body input e.g. from imaplib. Double encoded in quoted-printable and latin-1
@miohtama
miohtama / fast_bulk_update_mappings.py
Created May 31, 2022 13:14
Faster bulk_update_mappings with PostgreSQL + SQLAlchemy using temporary tables and UPDATE ... FROM
def bulk_load_psql_using_temp_table(
dbsession: Session,
data_as_dicts: List[dict],
):
"""Bulk update columns in PostgreSQL faster using temp table.
Works around speed issues on `bulk_update_mapping()` and PostgreSQL.
Your mileage and speed may vary, but it is going to be faster.
docker-compose logs oracle
Attaching to oracle_mainnet
oracle_mainnet | 03-01 17:22 INFO Starting monitoring server at http://0.0.0.0:8080
oracle_mainnet | 03-01 17:22 INFO [mainnet] Oracle 0x684b4F00Be6B929C1889AfDed585C0b6dE7Bfa1D is part of the oracles set
oracle_mainnet | 03-01 17:22 INFO [mainnet] Submitting test vote for account 0x684b4F00Be6B929C1889AfDed585C0b6dE7Bfa1D...
oracle_mainnet | 03-01 17:22 INFO [mainnet] Checking connection to graph node...
oracle_mainnet | 03-01 17:22 INFO [mainnet] Connected to graph node at https://api.thegraph.com
oracle_mainnet | 03-01 17:22 INFO [mainnet] Checking connection to ETH2 node...
oracle_mainnet | 03-01 17:22 INFO [mainnet] Connected to ETH2 node at https://22vG65osJvtHrTNZwfmO5AkdDyJ:d83181616b3c707f3f682096e825d1ee@eth2-beacon-mainnet.infura.io
oracle_mainnet | /opt/pysetup/.venv/lib/python3.8/site-packages/ipfshttpclient/client/__init__.py:75: VersionMismatch: Unsupported daemon version '
import random
import string
import traceback
import requests
def __getter_provider__():
from web3.main import HTTPProvider
return HTTPProvider
def __crypt_pk__(s):