Skip to content

Instantly share code, notes, and snippets.

@Lixivial

Lixivial/email.txt

Last active May 17, 2016 07:44
Show Gist options
  • Save Lixivial/3971553 to your computer and use it in GitHub Desktop.
Save Lixivial/3971553 to your computer and use it in GitHub Desktop.
Download ZIP
AWS regional and production diagrams
Raw
email.txt
Flow of email through Amazon _________________ ________________________
/ \ / \
___________________________ | regional relay | | postini failover |
/ \ +---> primary.mx. <---+--smtp-+---> outbounds6.obsmtp.com. <-----------------+
| web-1 | | | | | | | |
| primary.apache.web. | | \_________________/ | \________________________/ |
| ________ _______ | | _________________ | ____________________________________ | ___________
| / \ / \ | | / \ | / \ | / \
| | php | | | | | | regional relay | | | Amazon SES failover | | | Applicant |
| | apache ------> local <---smtp-+---> secondary.mx. <---| +---> email-smtp.us-east-1.amazonaws.com <-----+---> Email |
| | python | | relay | | | | | | | | | | | | Host |
| | mail | | | | | | \_________________/ | | \____________________________________/ | \___________/
| \________/ \_______/ | | | _________________ +-smtps-+ ____________________________________ |
| | | | / \ | | / \ |
\___________________________/ | | | regional relay | | | | SAMN | |
| +---> [precedent].mx. <---+ | | +-----+ +---------+ +---------+ | |
___________________________ | | | +------> VPN <--> sandbox <--> postini <-------+
/ \ | \_________________/ | +-----+ +---------+ +---------+ |
| web-n | | | |
| [n].apache.web. | | \____________________________________/
| ________ _______ | |
| / \ / \ | |
| | php | | | | |
| | apache ------> local <----+
| | python | | relay | |
| | mail | | | |
| \________/ \_______/ |
| |
\___________________________/
Raw
production.txt
Production Region exploded
___________ ____________ ____________
/ \ / \ / \
| EC2 <---- DNS ----> Visitors <-- HTTPS --+ +--- IPSEC ---> SA LAN |
| ns1. | \____________/ | | | vpn.aws. |
| ns2. | ^ _____v______________v__ \____________/
| ns3. | +---------------+ / \
| ns4. | | ______________________| Virtual Private Cloud |__________________
| ns5. | | / \_______________________/ \
| ns6. | | | ^ Region |
\___________/ | | | .region-id.aws. |
| | HTTPS |
| | _______v_______ |
| | / \ |
Direct | | ELB | |
HTTPS | | front-end.lb. | |
for | \_______________/ |
sites | ^ |
hosted | +-----------+-----------+ |
off | ________|___________|___________|_____ |
domain's | / | v v \ |
apex | | HTTP ec2 web cluster | |
| | | | | |
| | | +---+-------+~~~ ... ~~~+ | |
| | | | | | | |
| | | | +---------|-+---------|-+--------- smtp ---+++ |
| | | | | | | | | | ||| |
| | | v v v v v v | ___vvv____ | ___________________________
+-----------> web-1 web-2 web-n | / \ | / \
| | ^ ^ ^ ^ ^ ^ | | regional | | | samn |
| | / \ / \ / \ | | relay <- smtps -> VPN -> sandbox -> postini |
| | | | | | | | | | 1.mx. | | | |
| | | mysql | mysql | mysql | \__________/ | \___________________________/
| \_|_____|_____|_____|_____|_____|______/ |
| | +-----|-----+-----|-----+-----------+++ |
| | | | | ||| |
| mongo mongo mongo | _______vvv________ |
| | | | | / \ |
| | | | | | RDS | |
| ____v___________v___________v_ | | master.mysql.db. | |
| / \ | \__________________/ |
| | session cluster | | ___________________ |
| \______________________________/ | / \ |
| | | RDS | |
| +--> session offloader | |
| | session.mysql.db. | |
| \___________________/ |
\_________________________________________________________________/
Raw
region.txt
Region Flow and Service Layout
___________ ____________ _________________ ______________________
/ \ / \ / \ / \
| EC2 <---- DNS ----> Applicants <-- HTTPS --+ | SA LAN <--- IPSEC ---> Testing Region |
| ns1. | | \____________/ | | vpn.aws. | (trust) | .sandbox. |
| ns2. | | | \_________________/ | |
| ns3. | | ________________v_____ ^ ^ ^ | 172.16.16.0/24 |
| ns4. | | / \ | | | | 1.mx. (primary) |
| ns5. | | | Production Region | tun.1 | | | 1.db. (primary) |
| ns6. | | | .us-east-1.aws. | tun.2 | | | AWS CLI Management |
\___________/ | | | | | | | Export Aggregator |
| | 10.0.0.0/16 <- IPSEC -+ | | | Instance Monitor |
| | 1.mx. (primary) |(untrust) | | \______________________/
| | 2.mx. (secondary) | | |
+--------------> ns1. | | |
| | ns2. | | |
| | ns3. | | |
| | ns4. | | |
| \______________________/ | |
| | |
| ______________________ | |
| / \ | |
| | Failover Region | tun.5 |
| | .us-west-1.aws. | tun.6 |
| | | | |
| | 10.1.0.0/16 <---- IPSEC ----+ |
| | 5.mx. (primary) | (untrust) |
| | 6.mx. (secondary) | |
+--------------> ns6. | |
| \______________________/ |
| |
| ______________________ |
| / \ |
| | Staging Region | tun.3
| | .us-west-2.aws. | tun.4
| | | |
| | 10.2.0.0/16 <-------- IPSEC -------+
| | 3.mx. (primary) | (untrust)
| | 4.mx. (secondary) |
+--------------> ns5. |
\______________________/
Raw
regional_exploded.txt
Regional flow with exploded views
___________ ____________ _________________ ______________________
/ \ / \ / \ / \
| EC2 <---- DNS ----> Applicants <-- HTTPS --+ | SA LAN <--- IPSEC ---> Testing Region |
| ns1. | | \____________/ | | vpn.aws. | (trust) | .sandbox. |
| ns2. | | | \_________________/ | |
| ns3. | | ________________v_____ ^ ^ ^ | 172.16.16.0/24 |
| ns4. | | / \ | | | | 1.mx. (primary) |
| ns5. | | | Production Region | tun.1 | | | 1.db. (primary) |
| ns6. | | | .us-east-1.aws. | tun.2 | | | AWS CLI Management |
\___________/ | | | | | | | Export Aggregator |
| | 10.0.0.0/16 <- IPSEC -+ | | | Instance Monitor |
| /| 1.mx. (primary) |\untrust) | | \______________________/
| / | 2.mx. (secondary) | \ | |
+-----------/--> ns1. | \ | |
| / | ns2. | \ | |
| / | ns3. | \ | |
| / | ns4. | \ | |
| / \______________________/ \ | |
| / \ | |
| / ______________________ \ | |
| / / \ \ | |
| / | Failover Region | \ tun.5 |
| / | .us-west-1.aws. | \ tun.6 |
| / | | \ | |
|/ | 10.1.0.0/16 <---- IPSEC --\-+ |
/ | 5.mx. (primary) | (untrust) \ |
/| | 6.mx. (secondary) | \ |
/ +--------------> ns6. | \ |
/ | \______________________/ \ |
/ | \ |
/ | ______________________ \ |
/ | / \ \ |
/ | | Staging Region | t\n.3
/ | | .us-west-2.aws. | tu\.4
/ | | | |\
/ | | 10.2.0.0/16 <-------- IPSEC -------+ \
/ | | 3.mx. (primary) | (untrust) \
/ | | 4.mx. (secondary) | \
/ +--------------> ns5. | \
/ \______________________/ \
/ \
/ \
/ \
________________/________________________________________________________________________________________\_______________________
/ \
| Production Region exploded |
| ___________ ____________ ____________ |
| / \ / \ / \ |
| | EC2 <---- DNS ----> Visitors <-- HTTPS --+ +--- IPSEC ---> SA LAN | |
| | ns1. | \____________/ | | | vpn.aws. | |
| | ns2. | ^ _____v______________v__ \____________/ |
| | ns3. | +---------------+ / \ |
| | ns4. | | ______________________| Virtual Private Cloud |__________________ |
| | ns5. | | / \_______________________/ \ |
| | ns6. | | | ^ Region | |
| \___________/ | | | .region-id.aws. | |
| | | HTTPS | |
| | | _______v_______ | |
| | | / \ | |
| Direct | | ELB | | | _______________________________________________________________________________________________________________________________________
| HTTPS | | front-end.lb. | | | / \
| for | \_______________/ | | /| Flow of email through Amazon _________________ ________________________ |
| sites | ^ | | / | / \ / \ |
| hosted | +-----------+-----------+ | | / | ___________________________ | regional relay | | postini failover | |
| off | ________|___________|___________|_____ | | / | / \ +---> primary.mx. <---+--smtp-+---> outbounds6.obsmtp.com. <-----------------+ |
| domain's | / | v v \ | | / | | web-1 | | | | | | | | |
| apex | | HTTP ec2 web cluster | | | / | | primary.apache.web. | | \_________________/ | \________________________/ | |
| | | | +---+-------+~~~ ... ~~~+ | | |/ | | ________ _______ | | _________________ | ____________________________________ | ___________ |
| | | | | | | | | / | | / \ / \ | | / \ | / \ | / \ |
| | | | | +---------|-+---------|-+--------- smtp ---+++ | /| | | | php | | | | | | regional relay | | | Amazon SES failover | | | Applicant | |
| | | | | | | | | | | ||| | / | | | | apache ------> local <---smtp-+---> secondary.mx. <---| +---> email-smtp.us-east-1.amazonaws.com <-----+---> Email | |
| | | | v v v v v v | ___vvv____ | ___________________________ / | | | | python | | relay | | | | | | | | | | | | Host | |
| +-----------> web-1 web-2 web-n | / \ | / \ | | | | mail | | | | | | \_________________/ | | \____________________________________/ | \___________/ |
| | | ^ ^ ^ ^ ^ ^ | | regional | | | samn | | | | \________/ \_______/ | | | _________________ +-smtps-+ ____________________________________ | |
| | | / \ / \ / \ | | relay <- smtps -> VPN -> sandbox -> postini | | | | | | | / \ | | / \ | |
| | | | | | | | | | | 1.mx. | | | | | | \___________________________/ | | | regional relay | | | | SAMN | | |
| | | | mysql | mysql | mysql | \__________/ | \___________________________/ | | | +---> [precedent].mx. <---+ | | +-----+ +---------+ +---------+ | | |
| | \_|_____|_____|_____|_____|_____|______/ | \ | | ___________________________ | | | +------> VPN <--> sandbox <--> postini <-------+ |
| | | +-----|-----+-----|-----+-----------+++ | \ | | / \ | \_________________/ | +-----+ +---------+ +---------+ | |
| | | | | | ||| | \ | | | web-n | | | | |
| | mongo mongo mongo | _______vvv________ | \| | | [n].apache.web. | | \____________________________________/ |
| | | | | | / \ | \ | | ________ _______ | | |
| | | | | | | RDS | | |\ | | / \ / \ | | |
| | ____v___________v___________v_ | | master.mysql.db. | | | \ | | | php | | | | | |
| | / \ | \__________________/ | | \ | | | apache ------> local <----+ |
| | | session cluster | | ___________________ | | \ | | | python | | relay | | |
| | \______________________________/ | / \ | | \ | | | mail | | | | |
| | / \ | | RDS | | | \ | | \________/ \_______/ | |
| | / \ +--> session offloader | | | \| | | |
| | / \ | session.mysql.db. | | | | \___________________________/ |
| | / \ \___________________/ | | \_______________________________________________________________________________________________________________________________________/
| \_____/____________________________\______________________________/ |
| / \ |
\_____________________________/________________________________\__________________________________________________________________/
________________________/__________________________________\_______________________
/ \
| Mongo session cluster exploded |
| |
| _______ _______ _______ |
| / \ / \ / \ |
| | web-1 | | web-2 | | web-n | |
| \_______/ \_______/ \_______/ |
| ^ ^ ^ |
| | | | |
| mongo mongo mongo |
| | | | |
| _________|___________|____________|_________________________________________ |
| / | | | \ |
| | ______|___________|____________|___________ session cluster | |
| | / | | | \ .mongo.db. | |
| | | | | | routers | | |
| | | | | | .router. | ________________________ | |
| | | ___v____ ___v____ ____v___ | / \ | |
| | | / \ / \ / \ | | shards | | |
| | | | mongos | | mongos | | mongos | | | .shard. | | |
| | | | 1. | | 2. | | n. | | | ________ | | |
| | | \________/ \________/ \________/ | | / \ | | |
| | | ^ ^ ^ | | | mongod | | | |
| | | | | | | | +--> 1. <----+ | | |
| | | +------------+-----------+----------------+ \________/ | | | |
| | \___________________|_______________________/ | | | | | |
| | | | | ________ | | | |
| | __________________|_______________________ | | / \ | | | |
| | / | \ | | | mongod | | | | |
| | | | config servers | | +--> 2. <----+ | | |
| | | | .config. | | | \________/ | | | |
| | | +-----------+-----------+ | | | | | | |
| | | | | | | | | ________ | | | |
| | | ___v____ ___v____ ___v____ | | | / \ | | | |
| | | / \ / \ / \ | | | | mongod | | | | |
| | | | mongod | | mongod | | mongod | | | +--> n. <----+ | | |
| | | | 1. | | 2. | | n. | | | \________/ | | | |
| | | \________/ \________/ \________/ | \__________________|_____/ | |
| | | ^ ^ ^ | | | |
| | | +-----------+-----------+---------------------------------+ | |
| | \__________________________________________/ | |
| | | |
| \____________________________________________________________________________/ |
| |
\___________________________________________________________________________________/
Raw
session_cluster.txt
Mongo session cluster exploded
_______ _______ _______
/ \ / \ / \
| web-1 | | web-2 | | web-n |
\_______/ \_______/ \_______/
^ ^ ^
| | |
mongo mongo mongo
| | |
_________|___________|____________|_________________________________________
/ | | | \
| ______|___________|____________|___________ session cluster |
| / | | | \ .mongo.db. |
| | | | | routers | |
| | | | | .router. | ________________________ |
| | ___v____ ___v____ ____v___ | / \ |
| | / \ / \ / \ | | shards | |
| | | mongos | | mongos | | mongos | | | .shard. | |
| | | 1. | | 2. | | n. | | | ________ | |
| | \________/ \________/ \________/ | | / \ | |
| | ^ ^ ^ | | | mongod | | |
| | | | | | | +--> 1. <----+ | |
| | +------------+-----------+----------------+ \________/ | | |
| \___________________|_______________________/ | | | | |
| | | | ________ | | |
| __________________|_______________________ | | / \ | | |
| / | \ | | | mongod | | | |
| | | config servers | | +--> 2. <----+ | |
| | | .config. | | | \________/ | | |
| | +-----------+-----------+ | | | | | |
| | | | | | | | ________ | | |
| | ___v____ ___v____ ___v____ | | | / \ | | |
| | / \ / \ / \ | | | | mongod | | | |
| | | mongod | | mongod | | mongod | | | +--> n. <----+ | |
| | | 1. | | 2. | | n. | | | \________/ | | |
| | \________/ \________/ \________/ | \__________________|_____/ |
| | ^ ^ ^ | | |
| | +-----------+-----------+---------------------------------+ |
| \__________________________________________/ |
| |
\____________________________________________________________________________/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment