This is a gist with a set of helpful performance tricks and best practices that I've found on the internet.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package main | |
import ( | |
"fmt" | |
"log" | |
"flag" | |
"time" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Latency Comparison Numbers (~2012) | |
---------------------------------- | |
L1 cache reference 0.5 ns | |
Branch mispredict 5 ns | |
L2 cache reference 7 ns 14x L1 cache | |
Mutex lock/unlock 25 ns | |
Main memory reference 100 ns 20x L2 cache, 200x L1 cache | |
Compress 1K bytes with Zippy 3,000 ns 3 us | |
Send 1K bytes over 1 Gbps network 10,000 ns 10 us | |
Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD |
L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns
Compress 1K bytes with Zippy ............. 3,000 ns = 3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns = 20 µs
SSD random read ........................ 150,000 ns = 150 µs
Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs
adapted from the article "Crawling anonymously with Tor in Python" by S. Acharya, Nov 2, 2013.
The most common use-case is to be able to hide one's identity using TOR or being able to change identities programmatically, for example when you are crawling a website like Google and you don’t want to be rate-limited or blocked via IP address.
Install Tor.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Should work on all Debian based distros with systemd; tested on Ubuntu 16.04+. | |
# This will by default install all plugins; you can customize this behavior on line 6. Selecting too many plugins can cause issues when downloading. | |
# Run as root (or sudo before every line) please. Note this is not designed to be run automatically; I recommend executing this line by line. | |
apt install curl | |
curl https://getcaddy.com | bash -s personal dns,docker,dyndns,hook.service,http.authz,http.awses,http.awslambda,http.cache,http.cgi,http.cors,http.datadog,http.expires,http.filemanager,http.filter,http.forwardproxy,http.geoip,http.git,http.gopkg,http.grpc,http.hugo,http.ipfilter,http.jekyll,http.jwt,http.locale,http.login,http.mailout,http.minify,http.nobots,http.prometheus,http.proxyprotocol,http.ratelimit,http.realip,http.reauth,http.restic,http.upload,http.webdav,net,tls.dns.auroradns,tls.dns.azure,tls.dns.cloudflare,tls.dns.cloudxns,tls.dns.digitalocean,tls.dns.dnsimple,tls.dns.dnsmadeeasy,tls.dns.dnspod,tls.dns.dyn,tls. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Should work on all Debian based distros with systemd; tested on Ubuntu 16.04+. | |
# This will by default install all plugins; you can customize this behavior on line 6. Selecting too many plugins can cause issues when downloading. | |
# Run as root (or sudo before every line) please. Note this is not designed to be run automatically; I recommend executing this line by line. | |
apt install curl | |
curl https://getcaddy.com | bash -s personal dns,docker,dyndns,hook.service,http.authz,http.awses,http.awslambda,http.cache,http.cgi,http.cors,http.datadog,http.expires,http.filemanager,http.filter,http.forwardproxy,http.geoip,http.git,http.gopkg,http.grpc,http.hugo,http.ipfilter,http.jekyll,http.jwt,http.locale,http.login,http.mailout,http.minify,http.nobots,http.prometheus,http.proxyprotocol,http.ratelimit,http.realip,http.reauth,http.restic,http.upload,http.webdav,net,tls.dns.auroradns,tls.dns.azure,tls.dns.cloudflare,tls.dns.cloudxns,tls.dns.digitalocean,tls.dns.dnsimple,tls.dns.dnsmadeeasy,tls.dns.dnspod,tls.dns.dyn,tls. |
- Any allowed
id
andname
attributes have"user-content-"
prefixed to their value. - A maximum of 2 nested
sub
orsup
tags are allowed. - A maximum of 10 nested
ul
orol
tags are allowed.
Allowed Element | Allowed Attributes |
---|---|
All allowed elements | abbr , accept , accept-charset , accesskey , action , align , alt , axis , border , cellpadding , cellspacing , char , charoff , charset , checked , clear , color , cols , colspan , compact , coords , datetime , dir , disabled , enctype , for , frame , headers , height , hreflang , hspace , id , ismap , itemprop , label , lang , maxlength , media , method , multiple , name , nohref , noshade , nowrap , open , prompt , readonly , rev , rows , rowspan , rules , scope , selected , shape , size , span , start , summary , tabindex , title , type , usemap , valign , value , vspace , width |
a |
As above plus href (must be github-mac , github-windows , http , |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash -e | |
echo ' | |
PATH=$HOME/go/bin:$PATH | |
export GOPATH=$HOME | |
export CDPATH=.:$HOME/src/golang.org/x:$HOME/go/src:$HOME/src/github.com:$HOME/src/github.com/nf:$HOME/src/github.com/adg | |
export EDITOR=vim | |
' >> ~/.profile | |
sudo apt-get update |
OlderNewer