Skip to content

Instantly share code, notes, and snippets.

@kekru kekru/01nginx-tls-sni.md
Last active Nov 13, 2019

Embed
What would you like to do?
nginx TLS SNI routing, based on subdomain pattern

Nginx TLS SNI routing, based on subdomain pattern

Nginx can be configured to route to a backend, based on the server's domain name, which is included in the SSL/TLS handshake (Server Name Indication, SNI).
This works for http upstream servers, but also for other protocols, that can be secured with TLS.

prerequisites

  • at least nginx 1.15.9 to use variables in ssl_certificate and ssl_certificate_key.
  • check nginx -V for the following:
    ...
    TLS SNI support enabled
    ...
    --with-stream_ssl_module 
    --with-stream_ssl_preread_module

It works well with the nginx:1.15.9-alpine docker image.

non terminating, TLS pass through

Pass the TLS stream to an upstream server, based on the domain name from TLS SNI field. This does not terminate TLS.
The upstream server can serve HTTPS or other TLS secured TCP responses.

stream {  

  map $ssl_preread_server_name $targetBackend {
    ab.mydomain.com  upstream1.example.com:443;
    xy.mydomain.com  upstream2.example.com:443;
  }   
 
  server {
    listen 443; 
        
    proxy_connect_timeout 1s;
    proxy_timeout 3s;
    resolver 1.1.1.1;
    
    proxy_pass $targetBackend;       
    ssl_preread on;
  }
}

terminating TLS, forward TCP

Terminate TLS and forward the plain TCP to the upstream server.

stream {  

  map $ssl_server_name $targetBackend {
    ab.mydomain.com  upstream1.example.com:443;
    xy.mydomain.com  upstream2.example.com:443;
  }

  map $ssl_server_name $targetCert {
    ab.mydomain.com /certs/server-cert1.pem;
    xy.mydomain.com /certs/server-cert2.pem;
  }

  map $ssl_server_name $targetCertKey {
    ab.mydomain.com /certs/server-key1.pem;
    xy.mydomain.com /certs/server-key2.pem;
  }
  
  server {
    listen 443 ssl; 
    ssl_protocols       TLSv1.2;
    ssl_certificate     $targetCert;
    ssl_certificate_key $targetCertKey;
        
    proxy_connect_timeout 1s;
    proxy_timeout 3s;
    resolver 1.1.1.1;
      
    proxy_pass $targetBackend;
  } 
}

Choose upstream based on domain pattern

The domain name can be matched by a regex pattern, and extracted to variables. See regex_names.
This can be used to choose a backend/upstream based on the pattern of a (sub)domain. This is inspired by robszumski/k8s-service-proxy.

The following configuration extracts a subdomain into variables and uses them to create the upstream server name.

stream {  

  map $ssl_preread_server_name $targetBackend {
    ~^(?<app>.+)-(?<namespace>.+).mydomain.com$ $app-public.$namespace.example.com:8080;
  }
  ...
}

Your Nginx should be reachable over the wildcard subdomain *.mydomain.com.
A request to shop-staging.mydomain.com will be forwarded to shop-public.staging.example.com:8080.

K8s service exposing by pattern

In Kubernetes, you can use this to expose all services with a specific name pattern.
This configuration exposes all service which names end with -public.
A request to shop-staging-9999.mydomain.com will be forwarded to shop-public in the namespace staging on port 9999.
You will also need to update the resolver, see below.

stream {  

  map $ssl_preread_server_name $targetBackend {
    ~^(?<service>.+)-(?<namespace>.+)-(?<port>.+).mydomain.com$ $service-public.$namespace.svc.cluster.local:$port;
  }
  
  server {
    ...
    resolver kube-dns.kube-system.svc.cluster.local;
    ...
  }
}
@razorRun

This comment has been minimized.

Copy link

razorRun commented Aug 16, 2019

Do we need to have an SSL certificate on ab.mydomain.com in this case?

@kekru

This comment has been minimized.

Copy link
Owner Author

kekru commented Aug 16, 2019

Hi razorRun, yes you need a certificate on the nginx, when using "terminating TLS, forward TCP".

If you use "non terminating, TLS pass through", then you will need the certificate on the backend server, but not on the nginx.

In both cases the certificate must match ab.mydomain.com

@razorRun

This comment has been minimized.

Copy link

razorRun commented Aug 17, 2019

Kekru Thanks, mate for the reply

I have a small clarification if you don't mind
What I want to do is
vpn1.app.com ─┬─► nginx at 10.0.0.1 ─┬─► vpn1 at another-server-1
vpn2.app.com ─┤ ├─► vpn2 at another-server-2
wildcard.app.com ─┘ ─► Y

I have a wildcard(all the subdomains) pointed to a nginx server

The thing is I want to do a dynamic mapping and am not in control of another-server-X. If a client asks for a subdomain(not fixed) I will have to have a lookup table and map subdomain to the expected external server.

So do I have to add a SSL cert to my nginx server. will it give the SNI? I am getting a blank "-" at the moment.

Any help will be really handy.
Thanks in advance

Current Config

stream {
log_format basic '$remote_addr [$time_local] '
'$ssl_preread_server_name'
'$protocol $status $bytes_sent $bytes_received '
'$session_time';

access_log  /var/log/nginx/access.log basic;
error_log  /var/log/nginx/error.log debug;

#I will dynamically update the map section
map $ssl_preread_server_name $targetBackend {
sample.mydomain.com 32.23.232.32:3431;
xy.mydomain.com 44.23.342.32:3431;
}
server {
listen 80;
proxy_pass $targetBackend;
ssl_preread on;
proxy_connect_timeout 1s;
proxy_timeout 3s;
resolver 1.1.1.1;

}

}

@kekru

This comment has been minimized.

Copy link
Owner Author

kekru commented Aug 19, 2019

Hi razorRun,

  1. Does "vpn1 at another-server-1" have a certificate? If no, then you need a certificate at your nginx.
    Same for "vpn2 at another-server-2"
  2. The SNI comes from the client, during the TLS handshake between client and nginx.
  3. It is not guaranteed, that all clients send an SNI, but most HTTPS clients should do, including all browsers.
  4. Which protocol are you using, between client and nginx? HTTPS or another? If another, be sure, that it is build on top of TCP+TLS. Otherwise it won't work.
  5. Your current config listens on port 80. If your protocol is HTTPS, be sure to explicitly write it in your browser's address line, or your browser will send HTTP by default. Then you don't have an SNI. SNI is only for TLS secured connections.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.