Skip to content

Instantly share code, notes, and snippets.

Avatar

andre da palma andrefsp

  • Sliide
  • London - UK
View GitHub Profile
View grpc client SSL transport
package main
import (
"context"
"crypto/tls"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
@andrefsp
andrefsp / squid.inboud.conf
Created Jul 15, 2019
Squid Double Legged proxy
View squid.inboud.conf
# Recommended minimum configuration:
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
# Specify incoming networks of incoming traffic
acl localnet src 0.0.0.0/0 192.168.100.0/24 192.168.101.0/24 10.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
View Squid Allow All configuration
#Recommended minimum configuration:
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src 0.0.0.0/0 192.168.100.0/24 192.168.101.0/24
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
@andrefsp
andrefsp / parallel_url_fetch.go
Created Feb 12, 2016
Parallel url fetcher written in go
View parallel_url_fetch.go
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
)
func fetch_url(url string, responses *chan string) {
@andrefsp
andrefsp / parallel_url_fetch_aiohttp.py
Created Feb 10, 2016
This gist shows how to make a non-blocking parallel url fetcher using aiohttp and asyncio
View parallel_url_fetch_aiohttp.py
from datetime import datetime
import logging
import asyncio
from aiohttp import web
import requests
logger = logging.getLogger()
@asyncio.coroutine
@andrefsp
andrefsp / gevent_non_blocking.py
Last active Feb 9, 2016
Gevent non blocking socket
View gevent_non_blocking.py
import gevent
from gevent import monkey
monkey.patch_all()
import requests
def fetch_request(url):
print("fetching %s" % url)
requests.get(url)
print("Finish fetching %s" % url)
@andrefsp
andrefsp / pylibmc_concurrency_test.py
Last active Aug 29, 2015
Race condition on pylibmc
View pylibmc_concurrency_test.py
import argparse
import multiprocessing
import threading
from pylibmc.test import make_test_client
import random
NUM_THREADS = 2
NUM_PROCS = 4
NUM_SETS = 10000
@andrefsp
andrefsp / postgres_db_stats
Created Oct 22, 2013
Shell script to get per database stats
View postgres_db_stats
#!/bin/bash
case $1 in
pg_stat_activity)
sudo -u postgres psql -t -c "select $3 from pg_stat_activity where datname='$2' ;" | head -1 | sed s/\ \\+//g
;;
pg_stat_database)
sudo -u postgres psql -t -c "select $3 from pg_stat_database where datname='$2' ;" | head -1 | sed s/\ \\+//g
;;
pg_database_size)
View gist:5198848
## At the http level
map $http_cookie $is_secure {
default 0;
~SESS 1; # there's a session cookie (use SSL - authenticated user)
}
map $is_secure $not_secure {
1 0;
0 1;
}
View solr-performance.rst

A general rule of thumb we work with is:

  • Use the Oracle JVM, not OpenJDK. OpenJDK has a lot of issues with large amounts of memory
  • You'll want twice as much memory dedicated to the JVM as the index size is on disk. i.e. if your committed/optimized index is 20GB in size, you'll want 40GB of RAM + a little spare.
  • Use a good garbage collector, e.g.::
    JAVA_OPTS="${JAVA_OPTS} -XX:+UseConcMarkSweepGC -XX:+UseParNewGC"
  • Logging helps a LOT when shit is not working as expected::
    JAVA_OPTS="${JAVA_OPTS} -Xloggc:/var/log/tomcat6/log_GarbageCollection -XX:+PrintGCDetails -XX:+PrintGCTimeStamps"JAVA_OPTS="${JAVA_OPTS} -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/tomcat6/oom_log"

REMEMBER, CACHES NEED RAM. ALLOCATED MORE RAM TO THE JVM AS IS REQUIRED BASED ON YOUR CACHES.