-
-
Save tony-gutierrez/198988c34e020af0192bab543d35a62a to your computer and use it in GitHub Desktop.
# Dont forget to set the env variable "certdomain", and either fill in your email below or use an env variable for that too. | |
# Also note that this config is using the LetsEncrypt staging server, remove the flag when ready! | |
Resources: | |
sslSecurityGroupIngress: | |
Type: AWS::EC2::SecurityGroupIngress | |
Properties: | |
GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]} | |
IpProtocol: tcp | |
ToPort: 443 | |
FromPort: 443 | |
CidrIp: 0.0.0.0/0 | |
files: | |
# The Nginx config forces https, and is meant as an example only. | |
/etc/nginx/conf.d/000_http_redirect_custom.conf: | |
mode: "000644" | |
owner: root | |
group: root | |
content: | | |
server { | |
listen 8080; | |
return 301 https://$host$request_uri; | |
} | |
# The Nginx config forces https, and is meant as an example only. | |
/etc/nginx/conf.d/https_custom.pre: | |
mode: "000644" | |
owner: root | |
group: root | |
content: | | |
# HTTPS server | |
server { | |
listen 443 default ssl; | |
server_name localhost; | |
error_page 497 https://$host$request_uri; | |
ssl_certificate /etc/letsencrypt/live/ebcert/fullchain.pem; | |
ssl_certificate_key /etc/letsencrypt/live/ebcert/privkey.pem; | |
ssl_session_timeout 5m; | |
ssl_protocols TLSv1.1 TLSv1.2; | |
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; | |
ssl_prefer_server_ciphers on; | |
if ($ssl_protocol = "") { | |
rewrite ^ https://$host$request_uri? permanent; | |
} | |
location ~ ^/(lib/|img/) { | |
root /var/app/current/public; | |
access_log off; | |
} | |
location / { | |
proxy_pass http://nodejs; | |
proxy_set_header Connection ""; | |
proxy_http_version 1.1; | |
proxy_set_header Host $host; | |
proxy_set_header X-Real-IP $remote_addr; | |
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
proxy_set_header Upgrade $http_upgrade; | |
proxy_set_header Connection "upgrade"; | |
} | |
} | |
packages: | |
yum: | |
epel-release: [] | |
container_commands: | |
10_installcertbot: | |
command: "wget https://dl.eff.org/certbot-auto;chmod a+x certbot-auto" | |
20_getcert: | |
command: "sudo ./certbot-auto certonly --debug --non-interactive --email XXX@XXX.com --agree-tos --standalone --domains ${certdomain} --keep-until-expiring --pre-hook \"service nginx stop\" --staging" | |
30_link: | |
command: "ln -sf /etc/letsencrypt/live/${certdomain} /etc/letsencrypt/live/ebcert" | |
40_config: | |
command: "mv /etc/nginx/conf.d/https_custom.pre /etc/nginx/conf.d/https_custom.conf" |
Here is my solution which is combining from @tony-gutierrez, @krissrex, and @williamweckl: https://github.com/Archinowsk/konsti-server/tree/master/.ebextensions. Thanks!
- Use
container_commands
instead ofappdeploy/post/
hooks- Accept multiple domains
- Use
commands
to clean up prior hooks and nginx configs on deploy- Split to multiple files
- Cronjob for automatic certificate renewal
configdeploy/post/
hook to prevent recreation of default configs- www is redirected to non-www
@Archinowsk This is awesome work. Thanks so much for taking the time to share this. One question I have, why is the 03_nginx.config after the certificate generation? Wouldn't this need to be run and configured prior to the certbot verification process?
@Archinowsk This is awesome work. Thanks so much for taking the time to share this. One question I have, why is the 03_nginx.config after the certificate generation? Wouldn't this need to be run and configured prior to the certbot verification process?
Good question :D. My thought was that nginx config is referring to the certificate files -> create those first. Not completely sure if the order matters at that point.
From the EB Docs: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#linux-container-commands
You can use the container_commands key to execute commands that affect your application source code. Container commands run after the application and web server have been set up and the application version archive has been extracted, but before the application version is deployed. Non-container commands and other customization operations are performed prior to the application source code being extracted.
So looks like the order of the files does not matter, it is the type of operation the file describes that determines when it will execute in the EB deployment lifecycle. In short, your code should work fine. I am however still struggling to get this to work. Off to do more tinkering...
Did you ever manage to get @Archinowsk configs to work?
Still getting the following error No such file or directory:fopen('/etc/letsencrypt/live/ebcert/fullchain.pem','r')
I did end up getting it to work. It is worth noting I do have this config split into separate files, just adding them all at once here for simplicity. If you try and use this all in one file you may have issues since there are multiple files:
declarations in one script. Here is my concatenated config as one piece:
commands:
10_cleanup:
command: |
rm -f /opt/elasticbeanstalk/hooks/configdeploy/post/*
rm -f /etc/nginx/conf.d/*
container_commands:
20_remove_default_configs:
command: |
rm -f /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf
rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
container_commands:
10_install_certbot:
command: |
wget https://dl.eff.org/certbot-auto
mv certbot-auto /usr/local/bin/certbot-auto
chown root /usr/local/bin/certbot-auto
chmod 0755 /usr/local/bin/certbot-auto
20_configure_cert:
command: |
certbot_command="/usr/local/bin/certbot-auto certonly --webroot --webroot-path /var/www/html --debug --non-interactive --email ${CERT_EMAIL} --agree-tos --keep-until-expiring --expand"
for domain in $(echo ${CERT_DOMAINS} | sed "s/,/ /g")
do
certbot_command="$certbot_command -d $domain"
done
eval $certbot_command
30_link_cert:
command: |
domain="$( cut -d ',' -f 1 <<< "${CERT_DOMAINS}" )";
if [ -d /etc/letsencrypt/live ]; then
domain_folder_name="$(ls /etc/letsencrypt/live | sort -n | grep $domain | head -1)";
if [ -d /etc/letsencrypt/live/${domain_folder_name} ]; then
ln -sfn /etc/letsencrypt/live/${domain_folder_name} /etc/letsencrypt/live/ebcert
fi
fi
files:
"/etc/nginx/conf.d/000_http_redirect_custom.conf":
mode: "000644"
owner: root
group: root
content: |
server {
listen 8080;
server_name localhost;
location /.well-known {
# Certbot root
alias /var/www/html/.well-known;
}
location / {
return 301 https://$host$request_uri;
}
}
"/etc/nginx/conf.d/https_custom.conf":
mode: "000644"
owner: root
group: root
content: |
upstream nodejs {
server 127.0.0.1:8081;
keepalive 256;
}
# HTTPS server
server {
listen 443 ssl default;
server_name localhost;
error_page 497 https://$host$request_uri;
if ($host ~* ^www\.(.*)) {
set $host_without_www $1;
rewrite ^(.*) https://$host_without_www$1 permanent;
}
if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
set $year $1;
set $month $2;
set $day $3;
set $hour $4;
}
access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;
access_log /var/log/nginx/access.log main;
location / {
proxy_pass http://nodejs;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
gzip on;
gzip_comp_level 4;
gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
ssl_certificate /etc/letsencrypt/live/ebcert/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ebcert/privkey.pem;
ssl_session_timeout 5m;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_prefer_server_ciphers on;
if ($ssl_protocol = "") {
rewrite ^ https://$host$request_uri? permanent;
}
}
files:
# Elastic Beanstalk recreates the default configuration during every configuration deployment
"/opt/elasticbeanstalk/hooks/configdeploy/post/99_kill_default_nginx.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash -xe
rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
service nginx stop
service nginx start
files:
# Cron to renew cert
"/etc/cron.d/certbot_renew":
mode: "000644"
owner: root
group: root
content: |
@weekly root /usr/local/bin/certbot-auto renew
Hey,
This is still not working for me. When I run this then connection is refused on both HTTP and HTTPS. I've added these connections to the security group of the AWS beanstalk.
For some reason my /etc/nginx/nginx.conf
file didn't have include /etc/nginx/conf.d/*.conf;
anymore so my configs were not loaded. I ended up deleting the default file and creating it myself.
Full example: https://github.com/Archinowsk/konsti-server/tree/9de485e25ac2cf757d3e4b0b73ef662bda766847/.ebextensions
files:
"/etc/nginx/nginx.pre":
mode: "000644"
owner: root
group: root
content: |
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
port_in_redirect off;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
log_format healthd '$msec"$uri"$status"$request_time"$upstream_response_time"$http_x_forwarded_for';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/mime.types;
include /etc/nginx/conf.d/*.conf;
}
container_commands:
10_setup_nginx:
command: |
sudo rm -f /tmp/deployment/config/#etc#nginx#nginx.conf
sudo rm -f /etc/nginx/nginx.conf
sudo mv /etc/nginx/nginx.pre /etc/nginx/nginx.conf
sudo service nginx stop
sudo service nginx start
@Archinowsk Thanks so much! Worked like a charm!
@mjgall nginx is not finding the certificate at this location
/etc/letsencrypt/live/ebcert/fullchain.pem
so either it is not being generated, or it was generated in another location.
Any solution to this ? Facing the same problem
Here is something I have working for a python flask Apache beanstalk application: https://github.com/russau/renew-certs-flask
The project from @Archinowsk was a big help getting this done.
Here is mine:
- Symfony project
- HTTPD (Apache)
- In AWS working with ElasticBeanstalk env (using the function GetOptionSetting)
- With multiple domains (separated by
,
)
# Dont forget to set the env variable "HTTPS_CERT_DOMAIN", and either fill in your email below or use an env variable for that too.
# @source https://gist.github.com/cichondev/08056094a27cfcc7424afc3afcbf781c
packages:
yum:
epel-release: []
mod24_ssl : []
# ElasticBeanstalk 'env' settings
# WARNING : This part is only the .dist setting,
# if you want to update data, change it here (if needed) BUT ALSO in the ElasticBeanstalk console (EB -> Configuration -> Logiciels -> Propriétés de l'environnement)
option_settings:
aws:elasticbeanstalk:application:environment:
HTTPS_CERT_DOMAIN: your-domain.com,your-domain.eu-west-3.elasticbeanstalk.com'
HTTPS_CONTACT_EMAIL: 'admin@email.com'
Resources:
# Open the 443 port
sslSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]}
IpProtocol: tcp
ToPort: 443
FromPort: 443
CidrIp: 0.0.0.0/0
files:
# The Apache config forces http to https, REPLACE all {{HTTPS_CERT_DOMAIN}} by ONE domain name.
/etc/httpd/conf.d/http_redirect_template.pre:
mode: "000644"
owner: root
group: root
content: |
<VirtualHost *:80>
ServerName {{HTTPS_CERT_DOMAIN}}
ServerAlias www.{{HTTPS_CERT_DOMAIN}}
DocumentRoot "/var/www/html/public"
RewriteEngine on
RewriteCond %{SERVER_NAME} ={{HTTPS_CERT_DOMAIN}} [OR]
RewriteCond %{SERVER_NAME} =www.{{HTTPS_CERT_DOMAIN}}
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
# The Apache config https, REPLACE all {{HTTPS_CERT_DOMAIN}} by ONE domain name.
/etc/httpd/conf.d/https_custom_template.pre:
mode: "000644"
owner: root
group: root
content: |
# HTTPS server
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName {{HTTPS_CERT_DOMAIN}}
ServerAlias www.{{HTTPS_CERT_DOMAIN}}
DocumentRoot "/var/www/html/public"
RewriteEngine on
SSLCertificateFile /etc/letsencrypt/live/{{HTTPS_CERT_DOMAIN}}/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/{{HTTPS_CERT_DOMAIN}}/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
/tmp/letsencrypt/options-ssl-apache.conf:
mode: "000644"
owner: root
group: root
content: |
SSLEngine on
# Intermediate configuration, tweak to your needs
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
SSLHonorCipherOrder on
SSLOptions +StrictRequire
# Add vhost name to log entries:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" vhost_combined
LogFormat "%v %h %l %u %t \"%r\" %>s %b" vhost_common
#CustomLog /var/log/apache2/access.log vhost_combined
#LogLevel warn
#ErrorLog /var/log/apache2/error.log
# Always ensure Cookies have "Secure" set (JAH 2012/1)
#Header edit Set-Cookie (?i)^(.*)(;\s*secure)??((\s*;)?(.*)) "$1; Secure$3$4"
# Certbot (HTTPS) install
"/opt/elasticbeanstalk/hooks/appdeploy/post/100_install_cert_bot.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
echo "Installing CERTBOT.."
wget https://dl.eff.org/certbot-auto
mv certbot-auto /usr/local/bin/certbot-auto
chown root /usr/local/bin/certbot-auto
chmod 0755 /usr/local/bin/certbot-auto
echo "CERTBOT installed!"
# Certbot (HTTPS) config cert
"/opt/elasticbeanstalk/hooks/appdeploy/post/101_configure_cert.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
domains="`{"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "HTTPS_CERT_DOMAIN"}}`"
https_contact_email="`{"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "HTTPS_CONTACT_EMAIL"}}`"
echo "Configuring CERTBOT for domains: $domains";
stop httpd || true
certbot_command="/usr/local/bin/certbot-auto certonly --debug --non-interactive --email $https_contact_email --agree-tos --standalone"
echo "Removing OLD 000_http* (.config) files"
rm -Rf /etc/httpd/conf.d/000_http*
for domain in $(echo $domains | sed "s/,/ /g")
do
echo "=== $domain ==="
echo "** Setting up CERTBOT for $domain"
certbot_command_domain="$certbot_command --domains $domain"
echo "** Running CERTBOT for $domain.."
eval $certbot_command_domain
echo "CERTBOT configured!";
echo "** Creating HTTPS VirtualHost:443 for $domain.."
formatted_domain=$(echo $domain | sed "s/\./_/g")
sed -e "s/{{HTTPS_CERT_DOMAIN}}/$domain/g" /etc/httpd/conf.d/https_custom_template.pre > /etc/httpd/conf.d/000_https_custom_$formatted_domain.conf
echo "** Creating redirect HTTP to HTTPS for $domain.."
sed -e "s/{{HTTPS_CERT_DOMAIN}}/$domain/g" /etc/httpd/conf.d/http_redirect_template.pre > /etc/httpd/conf.d/000_http_redirect_$domain.conf
done
echo "Removing .config templates.."
rm -Rf /etc/httpd/conf.d/https_custom_template.pre
rm -Rf /etc/httpd/conf.d/http_redirect_template.pre
mv /tmp/letsencrypt/options-ssl-apache.conf /etc/letsencrypt/options-ssl-apache.conf
echo "Restarting HTTPD.."
start httpd
restart httpd
# Cron to renew cert
"/etc/cron.d/certbot_renew":
mode: "000644"
owner: root
group: root
content: |
@weekly root /usr/local/bin/certbot-auto renew
# Redo Jobs if server restarted from EB
"/opt/elasticbeanstalk/hooks/restartappserver/post/101_redo_certbot.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
echo "Server has been restart from EB"
/opt/elasticbeanstalk/hooks/appdeploy/post/101_configure_cert.sh
I've updated my project to work with the new Amazon Linux 2 Python 3.7 platform: https://github.com/russau/renew-certs-flask-nginx
There are new "platform hook" features to make this a bit easier: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/platforms-linux-extend.html
Following the script of @cavasinf, I get the following error in /var/log/eb-activity.log
:
StandaloneBindError: Problem binding to port 80: Could not bind to IPv4 or IPv6.
Please see the logfiles in /var/log/letsencrypt for more details.
Multiple "no such file or directory" errors follow, which I suppose are a result of the above error, which is the first. I am not sure if there is a problem with the file itself or this is unique to me.
Following the script of @cavasinf, I get the following error in
/var/log/eb-activity.log
:StandaloneBindError: Problem binding to port 80: Could not bind to IPv4 or IPv6. Please see the logfiles in /var/log/letsencrypt for more details.
@corei8 In my case I want the HTTP to redirect to the HTTPS.
But in your case you may disable the port :80 and only keep the :443
Can you go into your EC2 of EB, and check the SecurityGroup -> Inbound rules ?
You need to have at least :
Type | Protocole | Plage de ports | Source |
---|---|---|---|
HTTP | TCP | 80 | 0.0.0.0/0 |
HTTPS | TCP | 443 | 0.0.0.0/0 |
My solution @ https://github.com/HausCloud/AWS-ElasticBeanstalk-SSL
Additional support requests/bug reports are welcome.
@mjgall nginx is not finding the certificate at this location
/etc/letsencrypt/live/ebcert/fullchain.pem
so either it is not being generated, or it was generated in another location.Any solution to this ? Facing the same problem
@williamweckl I'm getting the following error when attempting to use your file - any insight?
[2020-01-19T21:06:38.844Z]` INFO [13932] - [Application update app-eddf-200119_150438@192/AppDeployStage1/AppDeployPostHook/105_restart_nginx.sh] : Activity execution failed, because: nginx: [emerg] BIO_new_file("/etc/letsencrypt/live/ebcert/fullchain.pem") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/ebcert/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file) nginx: configuration file /etc/nginx/nginx.conf test failed (ElasticBeanstalk::ExternalInvocationError) caused by: nginx: [emerg] BIO_new_file("/etc/letsencrypt/live/ebcert/fullchain.pem") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/ebcert/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file) nginx: configuration file /etc/nginx/nginx.conf test failed (Executor::NonZeroExitStatus)
Have you found a solution to this? Currently facing the same error.
Hello all. I have been trying to set this up. Although it seems that certbot downloads a certificate all well and that the custom files are put inside the nginx config.d directory and that nginx restarts properly...I still can't seem to be able to reach the API using Https... Actually now even http is broken. Any ideas?
- I have
listen 80;
as my EB is listening on 80 - I also have
proxy_pass http://docker;
and have also triedproxy_pass http://127.0.0.1:5000;
as I'm running a Java setup
Here is my current config:
Resources:
sslSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]}
IpProtocol: tcp
ToPort: 443
FromPort: 443
CidrIp: 0.0.0.0/0
files:
/etc/nginx/conf.d/000_http_redirect_custom.conf:
mode: "000644"
owner: root
group: root
content: |
server {
listen 80;
return 301 https://$host$request_uri;
}
/etc/nginx/conf.d/https_custom.pre:
mode: "000644"
owner: root
group: root
content: |
# HTTPS server
server {
listen 443 default ssl;
server_name localhost;
error_page 497 https://$host$request_uri;
ssl_certificate /etc/letsencrypt/live/ebcert/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ebcert/privkey.pem;
ssl_session_timeout 5m;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_prefer_server_ciphers on;
location / {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
packages:
yum:
epel-release: []
container_commands:
00_create_dir:
command: "mkdir -p /opt/certbot"
10_installcertbotandchmod:
command: "wget https://dl.eff.org/certbot-auto -O /opt/certbot/certbot-auto;chmod a+x /opt/certbot/certbot-auto"
20_getcert:
command: "sudo /opt/certbot/certbot-auto certonly --standalone --debug --non-interactive --email myemail@email.com --agree-tos --domains mydomain.com --expand --renew-with-new-domains --pre-hook \"service nginx stop\""
30_link:
command: "sudo ln -sf /etc/letsencrypt/live/mywebsite.com /etc/letsencrypt/live/ebcert"
40_config:
command: "mv /etc/nginx/conf.d/https_custom.pre /etc/nginx/conf.d/https_custom.conf"
50_ls_nginx_config:
command: "ls -la /etc/nginx/conf.d"
EDIT:
I was polling the nginx config directory. The script seems to be working fine and puts the files in the nginx directory. But after this finishes, just before the app starts, something totally resets nginx, the directory is deleted and put back in place. So those files get deleted and the config is not there anymore. Any ideas?
The solution from @HausCloud works perfectly. Although I got permissions denied error, I managed to fix it by following this issue.
Hey @DavidSanek, I'm quite new to EB and the associated stuff. Would you be kind enough to post your final solution of the setup you have in .ebextensions (incorporating @HausCloud certbot stuff)? Regards
Hey @mstarcevic, I've created a Gist with both files that I needed to setup certbot.
@DavidSanek Thank you very much. Appreciated.
@panagath Were you able to figure it out?
Hey @HausCloud, I used your solution in your Gist and it deployed successfully with no errors. But https doesn't seem to work. The http version works fine though. Is there a step after deployment that I'm missing?
@sinmarcus3 I got it working just last week. Let me go back over my notes later today and I'll let you know what exactly I did.
@sinmarcus3 Details as promised:
- Make sure that you are running on Amazon Linux 2.
- Make sure you do the eb labs download.
- Put the .platform folder at the same level as the .elasticbeanstalk folder.
- In .platform/hooks/postdeploy/00_ssl_setup_certbost.sh script, set the following variables manually:
CERTBOT_NAME
CERTBOT_EMAIL
CERTBOT_DOMAINS
***Note the original version of the script had these coming through as environment variables but the in the revised script, you set them manually. - In the .ebextensions folder, have the two config files as follows (you can call them what you want):
single_instance.config
option_settings:
- namespace: aws:elasticbeanstalk:environment
option_name: EnvironmentType
value: SingleInstance
https.config
Resources:
sslSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: {"Fn::GetAtt":["AWSEBSecurityGroup", "GroupId"]}
IpProtocol: tcp
ToPort: 443
FromPort: 443
CidrIp: 0.0.0.0/0
- Make sure that you deploy the .platform and .ebextensions folders, both at the top level of your zip.
- Run eb deploy
Thanks @mstarcevic. It seemed to deploy fine with no errors or issues. The logs even suggest the certificate was successfully enabled:
Congratulations! You have successfully enabled...
The http site works fine but the https still doesn't respond and refuses to connect with the following error: ERR_CONNECTION_REFUSED
@sinmarcus3 I can't remember anything else that I may have had to do, either in eb or on the AWS console side of things. My domain names are registered through Route 53 and I know I did a bit of stuffing around with Alias records and such. But I'm pretty sure that I also got it going with the environment URL directly. How are you checking? Postman?
Did you include the eb environment URL in the domain names CERTBOT variable? And what format did you use, assuming you did...
I think that I included IP addresses somewhere along the line but no longer. However, you did remind me of one other thing... In the corresponding EC2 instance in the incoming rules, I have a provision for port 443. I think that I had to put that in... in any case, it's there...
ie port 443, TCP, 0.0.0.0/0 (for the security group tied in with the eb). That's to go with ports 80 and 22...
@sinmarcus3 Can you curl localhost:80 and localhost:443? Check your config. You may have a load balancer attached.
@HausCloud It failed with connection refused on port 443. Have checked the config and there's no load balancer is attached.
@sinmarcus3 I am having the exact same issue as you. I can see that the SSL certificate was successfully enabled, but I am also getting connection refused on port 443.
@HausCloud I just stepped through it with a fresh sample app. The problem occurs when I input multiple domains in 00_ssl_setup_certbot.sh, i.e., CERTBOT_DOMAINS = "somedomain.ap-southeast-2.elasticbeanstalk.com,somedomain.co.nz,www.somedomain.co.nz". The code says the certificate is successfully installed and the http site will work, but the https version of the site will return a connection refused error.
@sinmarcus3 I'm inputting 2 domains and it still works.
@sinmarcus3 Want to set up a zoom call? Let's see if we can debug it.
When I SSH into my EC2 console and run telnet <my_ip_address> 443
, I am receiving connection refused. Any ideas?
@HausCloud I managed to get it to work by rebuilding the environment. I didn't make any other changes.
@sinmarcus3 Want to set up a zoom call? Let's see if we can debug it.
HausCloud I tried setting up https using your code. But I found that nothing sort of happens even the logging things. I have set up domain for single instance in the said variable still it didn't work. I even tried it with fresh environment setup. But no luck.
@sinmarcus3, @HausCloud and @jhaist I am having a similar issue, when I deploy my application HTTPS doesn't work while HTTP does. However, if I rebuild the environment then HTTPS and HTTPS redirect start working until I deploy again, which requires another rebuild.
same issue as @gavleavitt, when I use the solution from @HausCloud https stops working after CodePipeline deploy phase, rebuilding environment helps. Tried some things like removing the hook script after first run, but no luck. (I have single domain)
@EffyCoder @gavleavitt Can you post your EB setup so I can try to reproduce the issue?
@optimistiks Double check your config is a actually a single instance. See if the permissions_fix helps. Check your connected EC2 instance if the security settings are listening on 443. Otherwise, try a few curls in and out of the instance to debug.
@HausCloud @optimistiks @gavleavitt @sinmarcus3. Had a similar issue, but I sorted it out - found that postdeploy script was filling /etc/nginx/nginx.conf
with duplicate server_names_hash_bucket_size 192;
entries, creating an invalid nginx configuration. Had to ssh into the instance (as root), run the postdeploy script manually and check the conf with nginx -t
.
I've modified the postdeploy script as following (also commented out the 'Prevent certificate installation if not clean sample app' part).
...
HTTP_STRING='^http\s*{$'
NAME_LIMIT='http {\nserver_names_hash_bucket_size 192;\n'
SERVER_NAMES_HASH='nserver_names_hash_bucket_size 192;'
# Prevent replace if not clean sample app
if ! grep -Fxq "SERVER_NAMES_HASH" /etc/nginx/nginx.conf; then
# Increase size of string name for --domains (for default EB configs)
if ! sed -i "s/$HTTP_STRING/$NAME_LIMIT/g" /etc/nginx/nginx.conf; then
log_and_exit 'ERROR: Changing server name limit failed'
fi
fi
# # Prevent certificate installation if not clean sample app
# CERT_REGEX="Certificate Name:\s+$CERTBOT_NAME"
# if certbot certificates | grep -Ew "$CERT_REGEX"; then
# log_and_exit 'INFO: Certificate already installed.'
# fi
# Set up certificates
...
I almost cried when I saw the lock icon as I spent 2 days running Linux commands. Just wanted to thank you for all the solutions in this thread.
I followed the steps as described above and concerning the advice of @tbezemer.
The certificate is downloaded and installed, all steps fromcontainer_commands
have been executed successfully, but the nginx does not use my custom configuration - in fact, the nginx configuration seems to be resetted to a standard configuration.
I checked this by connecting via SSH to the beanstalk-server. During deployment of my application I can see there for a short time my custom config filehttps_custom.conf
in the folder/etc/nginx/conf.d
, but a short time later it is gone...Finally I discovered the solution for my problem - I had to modify the line
40_config
as follows (50_restartnginx
also seems not to be necessary, I could remove it without problem):40_config: command: "mv /etc/nginx/conf.d/https_custom.pre /var/elasticbeanstalk/staging/nginx/conf.d/https_custom.conf"
Your comment saved me. I was struggling for month. Trying out various configurations. But had no luck.
I tried using the container_commands
to setup the nginx configuration files but i couldn't find the files after successful deployment. After carrying out some research, I was pointed to this AWS documentation on how to override nginx configuration files. My folder structure is shown below.
~/workspace/my-app/
|-- .platform
| -- nginx
| -- nginx.conf
| -- conf.d
| -- https_custom.conf
|-- .ebextensions
| -- ssl.config
I also had issues with installing the epel using the yum package manager on Amazon Linux 2. So i tweaked the setup to use rpm to install epel.
Here is the content of my ssl.config
:
# Dont forget to set the env variable "DOMAIN_LINK" and either fill in your email below or set the env variable "EMAIL_LINK" for that too.
---
Resources:
sslSecurityGroupIngress:
Properties:
CidrIp: 0.0.0.0/0
FromPort: 443
GroupId:
? "Fn::GetAtt"
:
- AWSEBSecurityGroup
- GroupId
IpProtocol: tcp
ToPort: 443
Type: "AWS::EC2::SecurityGroupIngress"
files:
/etc/cron.d/certbot_renew:
content: "@weekly root certbot renew\n"
group: root
mode: "000644"
owner: root
container_commands:
10_downloadepel:
command: "sudo wget -r --no-parent -A 'epel-release-*.rpm' https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/"
20_installepel:
command: "sudo rpm -Uvh dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-*.rpm --force"
30_enableepl:
command: "sudo yum-config-manager --enable epel*"
40_installcertbot:
command: "sudo yum install -y certbot"
50_getcert:
command: "sudo certbot certonly --debug --non-interactive --email ${EMAIL_LINK} --agree-tos --standalone --domains ${DOMAIN_LINK} --keep-until-expiring --pre-hook \"sudo service nginx stop\" --post-hook \"sudo service nginx start\""
60_link:
command: "ln -sf /etc/letsencrypt/live/${DOMAIN_LINK} /etc/letsencrypt/live/ebcert"
Below is the content of nginx.conf
:
#Elastic Beanstalk Nginx Configuration File
user nginx;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 32137;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
include conf.d/*.conf;
map $http_upgrade $connection_upgrade {
default "upgrade";
}
server {
listen 80 default_server;
access_log /var/log/nginx/access.log main;
client_header_timeout 60;
client_body_timeout 60;
keepalive_timeout 60;
gzip off;
gzip_comp_level 4;
gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
# Include the Elastic Beanstalk generated locations
include conf.d/elasticbeanstalk/*.conf;
}
}
Below is the content of https_custom.conf
:
upstream nodejs {
server 127.0.0.1:3030;
keepalive 256;
}
# HTTPS server
server {
listen 443 default ssl;
server_name localhost;
error_page 497 https://$host$request_uri;
if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
set $year $1;
set $month $2;
set $day $3;
set $hour $4;
}
access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;
ssl_certificate /etc/letsencrypt/live/ebcert/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ebcert/privkey.pem;
ssl_session_timeout 5m;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_prefer_server_ciphers on;
if ($ssl_protocol = "") {
rewrite ^ https://$host$request_uri? permanent;
}
location ~ ^/(lib/|img/) {
root /var/app/current/public;
access_log off;
}
location / {
proxy_pass http://nodejs;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
Hopefully this helps.
I am getting the following error:
"Your system is not supported by certbot-auto anymore.
Certbot cannot be installed.
"
It looks like certbot-auto was deprecated:
certbot/certbot@cac9d8f
Here is a discussion:
https://community.letsencrypt.org/t/your-system-is-not-supported-by-certbot-auto-anymore/135504/21
Any other ideas about how to get it to work?
Thank you.
I almost cried when I saw the lock icon as I spent 2 days running Linux commands. Just wanted to thank you for all the solutions in this thread.
@morsanu Happy to help.
I am getting the following error:
"Your system is not supported by certbot-auto anymore.
Certbot cannot be installed.
"It looks like certbot-auto was deprecated:
certbot/certbot@cac9d8fHere is a discussion:
https://community.letsencrypt.org/t/your-system-is-not-supported-by-certbot-auto-anymore/135504/21Any other ideas about how to get it to work?
Thank you.
You can install the original certbot, instead of certbot-auto
wget -O epel.rpm –nv https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo yum install -y ./epel.rpm
sudo yum install -y python2-certbot-apache.noarch
then replace ./certbot-auto
command with certbot
Thank you so much @vahiwe, it worked flawlessly.
Thank you so much @vahiwe, it worked flawlessly.
@andylolu2 you're welcome. You can reference an article I wrote on it here.
@vahiwe I took the OP's post and combined it with your solution from (above) changed a few params to work with Apache and got this:
Resources:
sslSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]}
IpProtocol: tcp
ToPort: 443
FromPort: 443
CidrIp: 0.0.0.0/0
files:
/etc/httpd/conf.d/ssl.pre:
mode: "000644"
owner: root
group: root
content: |
LoadModule ssl_module modules/mod_ssl.so
Listen 443
<VirtualHost *:443>
<Directory /opt/python/current/app/build/static>
Order deny,allow
Allow from all
</Directory>
SSLEngine on
SSLCertificateFile "/etc/letsencrypt/live/${MY_DOMAIN}/fullchain.pem"
SSLCertificateKeyFile "/etc/letsencrypt/live/${MY_DOMAIN}/privkey.pem"
SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
SSLProtocol All -SSLv2 -SSLv3
SSLHonorCipherOrder On
SSLSessionTickets Off
Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
Header always set X-Frame-Options DENY
Header always set X-Content-Type-Options nosniff
ProxyPass / http://localhost:80/ retry=0
ProxyPassReverse / http://localhost:80/
ProxyPreserveHost on
RequestHeader set X-Forwarded-Proto "https" early
# If you have pages that may take awhile to
# respond, add a ProxyTimeout:
# ProxyTimeout seconds
</VirtualHost>
/tmp/renew_cert_cron:
mode: "000777"
owner: root
group: root
content: |
# renew Lets encrypt cert with certbot command
0 1,13 * * * /tmp/certbot-auto renew
packages:
yum:
epel-release: []
mod_ssl : []
container_commands:
10_downloadepel:
command: "sudo wget -r --no-parent -A 'epel-release-*.rpm' https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/"
20_installepel:
command: "sudo rpm -Uvh dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-*.rpm --force"
30_enableepl:
command: "sudo yum-config-manager --enable epel*"
40_installcertbot:
command: "sudo yum install -y certbot"
50_getcert:
command: "sudo certbot certonly --debug --non-interactive --email test@test.com --agree-tos --standalone --domains ${MY_DOMAIN} --keep-until-expiring --pre-hook \"sudo service httpd stop\" --post-hook \"sudo service httpd start\""
60_link:
command: "ln -sf /etc/letsencrypt/live/${MY_DOMAIN} /etc/letsencrypt/live/ebcert"
And it executes (all commends run) but https is still not working (http works fine, but https immediately returns 'Unable to connect') Any ideas? I've been stuck for 2 days now.
@zedy I haven’t used Apache in any of my projects so might not really know what works and what doesn’t. But we can try connecting and solving it together.
@vahiwe Thanks for the response. I figured it out. The ssl.pre
from the files section of the code block doesn't get executed and is never created, so i created it manually. Working perfectly. Thanks again.
Created ssl.conf
in /etc/httpd/conf.d/
(mod => 644, owner:group => root:root)
<VirtualHost *:443>
<Directory /opt/python/current/app/build/static>
Order deny,allow
Allow from all
</Directory>
SSLEngine on
SSLCertificateFile "/etc/letsencrypt/live/${MY_DOMAIN}/fullchain.pem"
SSLCertificateKeyFile "/etc/letsencrypt/live/${MY_DOMAIN}/privkey.pem"
SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
SSLProtocol All -SSLv2 -SSLv3
SSLHonorCipherOrder On
SSLSessionTickets Off
Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
Header always set X-Frame-Options DENY
Header always set X-Content-Type-Options nosniff
ProxyPass / http://localhost:80/ retry=0
ProxyPassReverse / http://localhost:80/
ProxyPreserveHost on
RequestHeader set X-Forwarded-Proto "https" early
# If you have pages that may take awhile to
# respond, add a ProxyTimeout:
# ProxyTimeout seconds
</VirtualHost>
p.s. don't forget to restart sudo service httpd restart
👍 @zedy. You can share for others who might have the same issue.
Maybe this will help someone:
I updated my single web instance PHP Platform version and all my https stuff broke because of certbot-auto being deprecated. I'm just using the httpd config files that beanstalk uses by default.
I spent a while trying to get 'certbot' installed by all the different solutions above and elsewhere but kept running into errors/issues.
I was finally able to get it all working using this .ebextension config: https://gist.github.com/caseypage/3f59f29f1fb4d6590c9193340a38ea03
Maybe this will help someone:
I updated my single web instance PHP Platform version and all my https stuff broke because of certbot-auto being deprecated. I'm just using the httpd config files that beanstalk uses by default.
I spent a while trying to get 'certbot' installed by all the different solutions above and elsewhere but kept running into errors/issues.
I was finally able to get it all working using this .ebextension config: https://gist.github.com/caseypage/3f59f29f1fb4d6590c9193340a38ea03
Thank You! I also ran into issues because of certbot-auto being deprecated and tried different solutions. I was about to give up on this approach until I saw this comment.
@vahiwe I can't thank you enough. Worked perfectly.
for those who is interested, i have created a simple template(for python and nodejs). the EB portion is generic and can be used for other framework(.NET etc.)
Anyone with a working example of using acme.sh?
If I'm not mistaken, this is all it takes now:
.ebextensions
|_ 00_epel.config
|_ 01_AWS_Single_LetsEncrypt.config
00_epel.config
commands:
add_epel_repo:
command: "sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm"
test: "[ ! -e /tmp/add_epel_repo_run_once ] && touch /tmp/add_epel_repo_run_once || exit 0"
ignoreErrors: true
01_AWS_Single_LetsEncrypt.config
Resources:
sslSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: { "Fn::GetAtt": ["AWSEBSecurityGroup", "GroupId"] }
IpProtocol: tcp
ToPort: 443
FromPort: 443
CidrIp: 0.0.0.0/0
packages:
yum:
epel-release: []
container_commands:
01_certbot_install:
command: "sudo yum install certbot python-certbot-nginx"
02_certbot_generate_certs:
command: "sudo certbot --agree-tos --non-interactive --domains ${CERT_DOMAINS} --email ${CERT_EMAIL} --nginx"
03_certbot_auto_renew:
command: "sudo certbot renew --dry-run"
This will not work, because container_commands
runs before Elastic Beanstalk deploys and runs your application and the proxy server. So, the /etc/nginx/nginx.conf
will be overridden.
The structure that works for me is shown below:
|-- .platform
| -- hooks
| -- postdeploy
| -- ssl_setup_certbot.sh
|-- .ebextensions
| -- ssl.config
The content of ssl_setup_certbot.sh:
#!/usr/bin/env bash
echo "Installing CERTBOT.."
CERT_DOMAIN=`/opt/elasticbeanstalk/bin/get-config environment -k CERT_DOMAIN`
CERT_EMAIL=`/opt/elasticbeanstalk/bin/get-config environment -k CERT_EMAIL`
sudo yum -y install certbot python-certbot-nginx
sudo certbot --agree-tos --non-interactive --domains ${CERT_DOMAIN} --email ${CERT_EMAIL} --nginx
sudo certbot renew --dry-run
echo "CERTBOT installed!"
The content of ssl.config:
Resources:
sslSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: { "Fn::GetAtt": ["AWSEBSecurityGroup", "GroupId"] }
IpProtocol: tcp
ToPort: 443
FromPort: 443
CidrIp: 0.0.0.0/0
packages:
yum:
epel-release: []
files:
"/etc/cron.d/certbot_renew":
mode: "000644"
owner: root
group: root
content: |
0 */12 * * * root /usr/bin/certbot -q renew --nginx
commands:
add_epel_repo:
command: "sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm"
test: "[ ! -e /tmp/add_epel_repo_run_once ] && touch /tmp/add_epel_repo_run_once || exit 0"
ignoreErrors: true
CERT_DOMAIN and CERT_EMAIL is environment variable included in the eb environment.
Thanks @lucas-coelho, I'll test that out
I pretty much got exhausted and switched to CodeDeploy 😅
Hi @lucas-coelho.
I'm trying this your config in code but not run https.
can you tell me what config am i missing?
Here is my solution which is combining from @tony-gutierrez, @krissrex, and @williamweckl: https://github.com/Archinowsk/konsti-server/tree/master/.ebextensions. Thanks!
container_commands
instead ofappdeploy/post/
hookscommands
to clean up prior hooks and nginx configs on deployconfigdeploy/post/
hook to prevent recreation of default configs// Edit: Some updates. Turns out cert challenge check was not working properly. Hopefully fixed now.