Skip to content

Instantly share code, notes, and snippets.

View YanhaoYang's full-sized avatar

Yanhao Yang YanhaoYang

15 followers · 7 following
  • Berlin, Germany
View GitHub Profile
@YanhaoYang
YanhaoYang / parse_log.rb
Created May 28, 2019 19:29
Parse Heroku logs and index the logs in Elasticsearch
require 'elasticsearch'
require 'time'
client = Elasticsearch::Client.new host: "172.17.0.1", log: true
mappings = {
mappings: {
log: {
properties: {
ts: { type: 'date' },
@YanhaoYang
YanhaoYang / crawler.rb
Created May 27, 2019 14:16
Create a crawler with Capybara
require 'capybara'
require 'capybara/dsl'
require 'selenium-webdriver'
# java -jar selenium-server-standalone-3.14.0.jar
#
if ENV["SELENIUM_REMOTE_HOST"]
Capybara.register_driver :selenium_chrome_remote do |app|
args = [
"--disable-infobars",
@YanhaoYang
YanhaoYang / run-jet.sh
Created May 25, 2019 08:27
Archive current repository and run Codeship's jet
#!/bin/bash
set -x
set -e
root_dir=$(pwd)
tmpdir=$(mktemp -d "${TMPDIR:-/tmp/}$(basename $0).XXXXXXXXXXXX")
git archive --prefix=my-jet/ HEAD > "$tmpdir/archive.tgz"
@YanhaoYang
YanhaoYang / aws-ftp-role.json
Created May 25, 2019 08:24
aws ftp role
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowListingOfUserFolder",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Effect": "Allow",
@YanhaoYang
YanhaoYang / delete-git-branches.sh
Created April 28, 2019 18:20
#!/bin/bash
delete_branches () {
branches=($(git branch | sed 's/*//g'))
element_count=${#branches[@]}
selected_branches=()
echo "Select branches:"
for br in "${branches[@]}"
do
echo "Delete $br ? y/(n)"
@YanhaoYang
YanhaoYang / notes-for-command-lines.sh
Last active May 25, 2019 08:29
Some notes for command lines
# Find all .orig files, and delete them with confirmation
find . -name "*.orig" && read -q "REPLY?Delete all?" && find . -name "*.orig" -delete && echo "\nDone."
# pg_restore with bandwidth control and progress bar
cat db.dump | pv -pera -s $(stat -c %s db.dump) -L 2000000 | pg_restore -d postgres://postgres:a@localhost/db_name --clean --no-owner --verbose -Fc
# check pg readiness
for i in {1..60}; do echo "Waiting for pg ..."; pg_isready -d $PG_URL && break || sleep 1; done
# http://www.grymoire.com/Unix/Awk.html
@YanhaoYang
YanhaoYang / notes-for-vim.vim
Last active April 13, 2019 09:41
Some notes for vim
" Copy from Vim to clipboard
vmap <leader>x :w! ~/.vbuf<cr>:!cat ~/.vbuf \| xclip -in -sel clip<cr><cr>
" Strip rspec outputs
:%s/rspec \(.\{-}\) #.\+/\1 \\/gc
@YanhaoYang
YanhaoYang / puma.rb
Created October 19, 2018 18:41
Print the call stacks in a multithread application, say puma, to know which threads are blocked
before_fork do
trap 'TTIN' do
Thread.list.each do |thread|
puts "\nThread TID-#{thread.object_id.to_s(36)}"
puts thread.backtrace.join("\n")
puts "\n"
end
end
end
@YanhaoYang
YanhaoYang / queries.sql
Created July 26, 2018 12:29
Some useful queries
SELECT nspname || '.' || relname AS "relation",
pg_size_pretty(pg_relation_size(C.oid)) AS "size"
FROM pg_class C
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
WHERE nspname NOT IN ('pg_catalog', 'information_schema')
ORDER BY pg_relation_size(C.oid) DESC
LIMIT 200 -- bigtables;
SELECT
pg_stat_activity.pid,
@YanhaoYang
YanhaoYang / docker-compose.yml
Created July 17, 2018 11:06
Docker compose with ssh-agent
version: '2'
services:
postgres:
image: postgres:9.6
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: a