A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."
- Does the design expect failures to happen regularly and handle them gracefully?
- Have we kept things as simple as possible?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/env bash | |
| # | |
| # Author: Stefan Buck | |
| # License: MIT | |
| # https://gist.github.com/stefanbuck/ce788fee19ab6eb0b4447a85fc99f447 | |
| # | |
| # | |
| # This script accepts the following parameters: | |
| # | |
| # * owner |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var request = require('request'); | |
| var unzip = require('unzip'); | |
| var csv2 = require('csv2'); | |
| request.get('http://s3.amazonaws.com/alexa-static/top-1m.csv.zip') | |
| .pipe(unzip.Parse()) | |
| .on('entry', function (entry) { | |
| entry.pipe(csv2()).on('data', console.log); | |
| }) | |
| ; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // Here is an extremely simple version of work scheduling for multiple | |
| // processors. | |
| // | |
| // The Problem: | |
| // We have a lot of numbers that need to be math'ed. Doing this on one | |
| // CPU core is slow. We have 4 CPU cores. We would thus like to use those | |
| // cores to do math, because it will be a little less slow (ideally | |
| // 4 times faster actually). | |
| // | |
| // The Solution: |
Here are the simple steps needed to create a deployment from your local GIT repository to a server based on this in-depth tutorial.
You are developing in a working-copy on your local machine, lets say on the master branch. Most of the time, people would push code to a remote server like github.com or gitlab.com and pull or export it to a production server. Or you use a service like deepl.io to act upon a Web-Hook that's triggered that service.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ############################################################################### | |
| # Helpful Docker commands and code snippets | |
| ############################################################################### | |
| ### CONTAINERS ### | |
| docker stop $(docker ps -a -q) #stop ALL containers | |
| docker rm -f $(docker ps -a -q) # remove ALL containers | |
| docker rm -f $(sudo docker ps --before="container_id_here" -q) # can also filter | |
| # exec into container |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Rules for transparent proxying | |
| iptables -N NO_PROXY -t nat | |
| iptables -A NO_PROXY -t nat -d 0.0.0.0/8 -j ACCEPT | |
| iptables -A NO_PROXY -t nat -d 10.0.0.0/8 -j ACCEPT | |
| iptables -A NO_PROXY -t nat -d 127.0.0.0/8 -j ACCEPT | |
| iptables -A NO_PROXY -t nat -d 169.254.0.0/16 -j ACCEPT | |
| iptables -A NO_PROXY -t nat -d 172.16.0.0/12 -j ACCEPT | |
| iptables -A NO_PROXY -t nat -d 192.168.0.0/16 -j ACCEPT | |
| iptables -A NO_PROXY -t nat -d 224.0.0.0/4 -j ACCEPT | |
| iptables -A NO_PROXY -t nat -d 240.0.0.0/4 -j ACCEPT |
NewerOlder
