Skip to content

Instantly share code, notes, and snippets.

@amitsaxena
Created November 6, 2019 02:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save amitsaxena/5e1603bdcbc7445d7308f61262a9646f to your computer and use it in GitHub Desktop.
Save amitsaxena/5e1603bdcbc7445d7308f61262a9646f to your computer and use it in GitHub Desktop.
How to clone a website and host it on a domain of your choice on AWS S3
# Install wget and AWS CLI and set up your AWS credentials
# STEP 1
# Create a mirror of the website and re-write links
# --wait specifies that there is a wait of 1 second after each request.
# Be gentle with someone else's server. Also you risk being blocked if your request rate is too high
# Use -U to spoof user agent to that of an actual browser in case the server is blocking you from making requests
wget --mirror --convert-links --html-extension --wait=1 -o log https://aawaara.com/
# STEP 2
# Create S3 bucket and enable "Static Website Hosting" for the bucket and also allow making files public (it's a bucket setting)
# For the given example create bucket badd431a8703451fa409d73d2218b545.aawaara.com
# Specify Index file as index.html
# STEP 3
# Upload files to S3 bucket using AWS CLI and make them public
cd aawaara.com
aws s3 sync . s3://badd431a8703451fa409d73d2218b545.aawaara.com/ --acl public-read
# STEP 4
# Add a CNAME entry in your DNS records which points to your S3 bucket
# You can tweak it as per your needs: https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html
# In my case I need the following CNAME record:
# CNAME badd431a8703451fa409d73d2218b545.aawaara.com badd431a8703451fa409d73d2218b545.aawaara.com.s3-website-us-east-1.amazonaws.com
# Please note the name of your S3 bucket should match your CNAME for this to work out of the box
@amitsaxena
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment