Skip to content

Instantly share code, notes, and snippets.

View miohtama's full-sized avatar
🏠
https://tradingstrategy.ai

Mikko Ohtamaa miohtama

🏠
https://tradingstrategy.ai
View GitHub Profile
@miohtama
miohtama / rollingwindow.py
Last active January 15, 2022 21:20
Rolling window rate limitation implementation for Pyramid
"""Rolling time window counter and rate limit using Redis.
Use Redis sorted sets to do a rolling time window counters and limiters. These are useful for preventing denial of service, flood and reputation attack against site elements which trigegr outgoing action (email, SMS).
Example how to do a Colander validator which checks that the form has not been submitted too many times within the time period::
import colander as s
@c.deferred
def throttle_invites_validator(node, kw):
@miohtama
miohtama / geth-secure.md
Last active January 13, 2022 23:11
Secure RPC connections to geth daemon

Go Ethereum (geth) is a software for Ethereum. geth doesn't provide secure networking and it should do this, as this kind of built-in functionality would increase complexity and add attack surface to critical blockchain node software. Fortunately, in UNIX world, you can easily combine different tools to work together. The solution to this particular problem is to use VPN/tunneling software for secure connections. The tunnel will expose the server local connections to your own computer. The most popular tool for this (available in every OS by default, nowadays including Windows) is [Secure Shell (SSH)][1].

Note this question only addresses issues how to

[If you are not familiar with SSH please first read SSH tutorial how to safely do passwordless logins using SSH keys][2].

Start a node on server. When the node starts it binds its RPC port to localhost (127.0.0.1 in IPv4, ::1 in IPv6). This is so-called loopback connection that you can only access from the computer itself and not from external netwo

@miohtama
miohtama / indexnow.py
Created January 11, 2022 08:35
Bing IndexNow URL submitter for Python
"""Use IndexNow API to send the latest updates to Bing.
Ensures Bing picks up the pages that should be fresh and interesting.
Do not wait them picked up by the natural crawler process.
To submit the pages you need to have a Bing known file at the root of your website.
https://www.indexnow.org/faq
IndexNow will also submit the Yandex.
@miohtama
miohtama / parse-hash-bang-arguments-in-javascript.js
Created January 6, 2012 12:08
Parse hash bang HTTP GET query style arguments from an URL using Javascript
/**
* Parse hash bang parameters from a URL as key value object.
*
* For repeated parameters the last parameter is effective.
*
* If = syntax is not used the value is set to null.
*
* #x&y=3 -> { x:null, y:3 }
*
* @param aURL URL to parse or null if window.location is used
@miohtama
miohtama / transaction-helper.component.ts
Created May 26, 2020 16:39
TypeScript + Angular Ethereum transaction progress bar component
import { Component, OnInit, Input, ViewChild, Output, EventEmitter } from '@angular/core';
import { Web3Service, WalletState, WalletType } from '../web3.service';
import Web3 from 'web3';
import { waitTransaction, isSuccessfulTransaction } from '../transactionwait';
import { isRequired, checkRequired } from '../requiredInput';
import { EthereumProgressBarComponent } from '../ethereum-progress-bar/ethereum-progress-bar.component';
import { Subscription } from 'rxjs';
import { NGXLogger } from 'ngx-logger';
// Called before creating a tranaction.
@miohtama
miohtama / find-long-queries.sql
Created December 2, 2021 17:39
PostgreSQL find long queries
SELECT
application_name,
pid,
now() - pg_stat_activity.query_start AS duration,
query,
state
FROM pg_stat_activity
WHERE (now() - pg_stat_activity.query_start) > interval '5 minutes';
@miohtama
miohtama / pgdash-loop.sh
Created December 2, 2021 17:30
Bash script for pgmetrics + pgdash reporting loop
#!/bin/bash
#
# Monitor loop for TimescaleDB using pgdash
# - TimescaleDB runs in a Docker, exposed over TCP/IP socket 127.0.0.1:5555
# - pgdash SaaS offering used https://pgdash.io
# - run the sample collection loop every 30 seconds
#
#
set -e
set -x
@miohtama
miohtama / dramatiq-task-queue.py
Last active November 6, 2021 01:47
Throttling background task queue using Dramatiq - wait until more workers are freed
import time
from typing import Dict, Optional
from dramatiq import Message, Broker
from dramatiq.results import ResultMissing
class TaskQueue:
"""A task queue using Dramatiq background task framework.
@miohtama
miohtama / bsh-notes.bash
Created November 4, 2021 18:37
Binance Smart Chain notes
./geth_linux \
--config ./config.toml \
--datadir ./data/bsc \
--cache 8000 \
--rpc.allow-unprotected-txs \
--txlookuplimit 0 \
--http.port 9545 \
--http.addr 127.0.0.1 \
--http.vhosts=* \
@miohtama
miohtama / gist:34a83d870a14aa7e580d
Last active October 31, 2021 13:21
Safe evaluation of math expressions in Python, using byte code verifier and eval()
""""
The orignal author: Alexer / #python.fi
"""
import opcode
import dis
import sys
import multiprocessing