Skip to content

Instantly share code, notes, and snippets.

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/37.0.2062.94 Chrome/37.0.2062.94 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/600.8.9 (KHTML, like Gecko) Version/8.0.8 Safari/600.8.9
Mozilla/5.0 (iPad; CPU OS 8_4_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12H321 Safari/600.1.4
Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240
Mozilla/5.0 (Windows NT 6.3; WOW64; rv:40.0)
@rgreenjr
rgreenjr / postgres_queries_and_commands.sql
Last active April 23, 2024 19:14
Useful PostgreSQL Queries and Commands
-- show running queries (pre 9.2)
SELECT procpid, age(clock_timestamp(), query_start), usename, current_query
FROM pg_stat_activity
WHERE current_query != '<IDLE>' AND current_query NOT ILIKE '%pg_stat_activity%'
ORDER BY query_start desc;
-- show running queries (9.2)
SELECT pid, age(clock_timestamp(), query_start), usename, query
FROM pg_stat_activity
WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%'
@sirodoht
sirodoht / migrate-django.md
Last active April 20, 2024 09:52
How to migrate Django from SQLite to PostgreSQL

How to migrate Django from SQLite to PostgreSQL

Dump existing data:

python3 manage.py dumpdata > datadump.json

Change settings.py to Postgres backend.

Make sure you can connect on PostgreSQL. Then:

@valyala
valyala / README.md
Last active April 19, 2024 13:00
Optimizing postgresql table for more than 100K inserts per second

Optimizing postgresql table for more than 100K inserts per second

  • Create UNLOGGED table. This reduces the amount of data written to persistent storage by up to 2x.
  • Set WITH (autovacuum_enabled=false) on the table. This saves CPU time and IO bandwidth on useless vacuuming of the table (since we never DELETE or UPDATE the table).
  • Insert rows with COPY FROM STDIN. This is the fastest possible approach to insert rows into table.
  • Minimize the number of indexes in the table, since they slow down inserts. Usually an index on time timestamp with time zone is enough.
  • Add synchronous_commit = off to postgresql.conf.
  • Use table inheritance for fast removal of old data:
@lukehedger
lukehedger / ffmpeg-compress-mp4
Last active April 18, 2024 21:00
Compress mp4 using FFMPEG
$ ffmpeg -i input.mp4 -vcodec h264 -acodec mp2 output.mp4
// XPath CheatSheet
// To test XPath in your Chrome Debugger: $x('/html/body')
// http://www.jittuu.com/2012/2/14/Testing-XPath-In-Chrome/
// 0. XPath Examples.
// More: http://xpath.alephzarro.com/content/cheatsheet.html
'//hr[@class="edge" and position()=1]' // every first hr of 'edge' class
@wwek
wwek / httpsproxy.go
Created October 29, 2017 05:41
https proxy in golang
// https://medium.com/@mlowicki/http-s-proxy-in-golang-in-less-than-100-lines-of-code-6a51c2f2c38c
// #!/usr/bin/env bash
// case `uname -s` in
// Linux*) sslConfig=/etc/ssl/openssl.cnf;;
// Darwin*) sslConfig=/System/Library/OpenSSL/openssl.cnf;;
// esac
// openssl req \
// -newkey rsa:2048 \
// -x509 \
@JeffBelback
JeffBelback / docker-destroy-all.sh
Last active December 12, 2023 17:47
Destroy all Docker Containers and Images
#!/bin/bash
# Stop all containers
containers=`docker ps -a -q`
if [ -n "$containers" ] ; then
docker stop $containers
fi
# Delete all containers
containers=`docker ps -a -q`
if [ -n "$containers" ]; then
docker rm -f -v $containers
@snakers4
snakers4 / parse_cc_index.py
Last active September 14, 2023 20:00
Plain common crawl pre-processing
import gc
import gzip
import time
import json
import shutil
import os,sys
import tldextract
import collections
import pandas as pd
from tqdm import tqdm