-
-
Save apollo13/857ae4c5e18de619815c2628212449e1 to your computer and use it in GitHub Desktop.
# Simple example to deploy traefik with consul connect enabled. | |
# For simplicity the job includes traefik as well as the backend service. | |
# Please note that traefik currently only supports connect for HTTP. | |
job "traefik-consul-connect-demo" { | |
datacenters = ["dc1"] | |
group "edge" { | |
network { | |
mode = "bridge" | |
port "http" { | |
to = 8080 | |
} | |
} | |
service { | |
name = "traefik-ingress" | |
port = "http" | |
connect { | |
native = true | |
} | |
} | |
task "traefik" { | |
driver = "docker" | |
config { | |
image = "traefik:v2.5.2" | |
args = [ | |
# Enables connect support, otherwise only http connections would be tried | |
"--providers.consulcatalog.connectaware=true", | |
# Make the communication secure by default | |
"--providers.consulcatalog.connectbydefault=true", | |
"--providers.consulcatalog.exposedbydefault=false", | |
"--entrypoints.http=true", | |
"--entrypoints.http.address=:8080", | |
# The service name below should match the nomad/consul service above | |
# and is used for intentions in consul | |
"--providers.consulcatalog.servicename=traefik-ingress", | |
"--providers.consulcatalog.prefix=traefik", | |
# Automatically configured by Nomad through CONSUL_* environment variables | |
# as long as client consul.share_ssl is enabled | |
# "--providers.consulcatalog.endpoint.address=<socket|address>" | |
# "--providers.consulcatalog.endpoint.tls.ca=<path>" | |
# "--providers.consulcatalog.endpoint.tls.cert=<path>" | |
# "--providers.consulcatalog.endpoint.tls.key=<path>" | |
# "--providers.consulcatalog.endpoint.token=<token>" | |
] | |
} | |
env { | |
# Enable this if nomad is older than 1.1.3 | |
# CONSUL_TLS_SERVER_NAME = "localhost" | |
} | |
} | |
} | |
group "backend" { | |
network { | |
mode = "bridge" | |
} | |
service { | |
name = "whoami" | |
port = 80 | |
tags = [ | |
"traefik.enable=true", | |
"traefik.http.routers.whoami.rule=Host(`whoami.example.com`)" | |
] | |
connect { | |
sidecar_service {} | |
} | |
} | |
# Note: For increased security the service should only listen on localhost | |
# Otherwise it could be reachable from the outside world without going through connect | |
task "whoami" { | |
driver = "docker" | |
config { | |
image = "containous/whoami" | |
} | |
} | |
} | |
} |
It seems that this requirement makes it impossible to route multiple service via that traefik instance, am I right ?
What makes you say that?
It seems that this requirement makes it impossible to route multiple service via that traefik instance, am I right ?
What makes you say that?
I want to use multiple services using different entrypoints, which means multiple service
stanza, I don't understand how to make that possible using the service-mesh integration.
ie I can configure multiple service stanza, but only one can be exposed to consul-catalog via service-name.
The different entrypoint
is probably what is missing from my first comment
Here is a job spec for reference: basically adding one more entrypoint and duplicating the backend section:
I could not make both the http1-ingress-demo
and http2-ingress-demo
working at the same time.
job "ingress-connect-demo" {
datacenters = ["meow"]
group "edge" {
network {
mode = "bridge"
port "http1" {
static = 8901
to = 8901
}
port "http2" {
static = 8902
to = 8902
}
}
service {
name = "http1-ingress-demo"
port = "http1"
connect {
native = true
}
}
service {
name = "http2-ingress-demo"
port = "http2"
connect {
native = true
}
}
task "traefik" {
driver = "docker"
config {
image = "traefik:2.6"
args = [
# Enables connect support, otherwise only http connections would be tried
"--providers.consulcatalog.connectaware=true",
# Make the communication secure by default
"--providers.consulcatalog.connectbydefault=true",
"--providers.consulcatalog.exposedbydefault=false",
"--entrypoints.http1=true",
"--entrypoints.http1.address=:8901",
"--entrypoints.http2=true",
"--entrypoints.http2.address=:8902",
# The service name below should match the nomad/consul service above
# and is used for intentions in consul
"--providers.consulcatalog.servicename=http1-ingress-demo",
# "--providers.consulcatalog.servicename=http2-ingress-demo",
"--providers.consulcatalog.prefix=ingress-demo",
# Automatically configured by Nomad through CONSUL_* environment variables
# as long as client consul.share_ssl is enabled
# "--providers.consulcatalog.endpoint.address=<socket|address>"
# "--providers.consulcatalog.endpoint.tls.ca=<path>"
# "--providers.consulcatalog.endpoint.tls.cert=<path>"
# "--providers.consulcatalog.endpoint.tls.key=<path>"
# "--providers.consulcatalog.endpoint.token=<token>"
]
}
env {
# Enable this if nomad is older than 1.1.3
# CONSUL_TLS_SERVER_NAME = "localhost"
}
}
}
group "backend1" {
network {
mode = "bridge"
}
service {
name = "whoami1"
port = 80
tags = [
"ingress-demo.enable=true",
"ingress-demo.http.routers.whoami1.rule=Host(`whoami1.example.com`)",
"ingress-demo.http.routers.whoami1.entrypoints=http1"
]
connect {
sidecar_service {}
}
}
# Note: For increased security the service should only listen on localhost
# Otherwise it could be reachable from the outside world without going through connect
task "whoami" {
driver = "docker"
config {
image = "containous/whoami:v1.5.0"
}
}
}
group "backend2" {
network {
mode = "bridge"
}
service {
name = "whoami2"
port = 80
tags = [
"ingress-demo.enable=true",
"ingress-demo.http.routers.whoami2.rule=Host(`whoami2.example.com`)",
"ingress-demo.http.routers.whoami2.entrypoints=http2"
]
connect {
sidecar_service {}
}
}
# Note: For increased security the service should only listen on localhost
# Otherwise it could be reachable from the outside world without going through connect
task "whoami" {
driver = "docker"
config {
image = "containous/whoami:v1.5.0"
}
}
}
}
If you have a way to make it work, I take it :)
That being said, if you use the same entrypoint for both backends, and route differently based on smth (like HOST header) it works as expected.
job "ingress-connect-demo" {
datacenters = ["meow"]
group "edge" {
network {
mode = "bridge"
port "http1" {
static = 8901
to = 8901
}
}
service {
name = "http1-ingress-demo"
port = "http1"
connect {
native = true
}
}
task "traefik" {
driver = "docker"
config {
image = "traefik:2.6"
args = [
# Enables connect support, otherwise only http connections would be tried
"--providers.consulcatalog.connectaware=true",
# Make the communication secure by default
"--providers.consulcatalog.connectbydefault=true",
"--providers.consulcatalog.exposedbydefault=false",
"--entrypoints.http1=true",
"--entrypoints.http1.address=:8901",
# The service name below should match the nomad/consul service above
# and is used for intentions in consul
"--providers.consulcatalog.servicename=http1-ingress-demo",
"--providers.consulcatalog.prefix=ingress-demo",
# Automatically configured by Nomad through CONSUL_* environment variables
# as long as client consul.share_ssl is enabled
# "--providers.consulcatalog.endpoint.address=<socket|address>"
# "--providers.consulcatalog.endpoint.tls.ca=<path>"
# "--providers.consulcatalog.endpoint.tls.cert=<path>"
# "--providers.consulcatalog.endpoint.tls.key=<path>"
# "--providers.consulcatalog.endpoint.token=<token>"
]
}
env {
# Enable this if nomad is older than 1.1.3
# CONSUL_TLS_SERVER_NAME = "localhost"
}
}
}
group "backend1" {
network {
mode = "bridge"
}
service {
name = "whoami1"
port = 80
tags = [
"ingress-demo.enable=true",
"ingress-demo.http.routers.whoami1.rule=Host(`whoami1.example.com`)",
"ingress-demo.http.routers.whoami1.entrypoints=http1"
]
connect {
sidecar_service {}
}
}
# Note: For increased security the service should only listen on localhost
# Otherwise it could be reachable from the outside world without going through connect
task "whoami" {
driver = "docker"
config {
image = "containous/whoami:v1.5.0"
}
}
}
group "backend2" {
network {
mode = "bridge"
}
service {
name = "whoami2"
port = 80
tags = [
"ingress-demo.enable=true",
"ingress-demo.http.routers.whoami2.rule=Host(`whoami2.example.com`)",
"ingress-demo.http.routers.whoami2.entrypoints=http1"
]
connect {
sidecar_service {}
}
}
# Note: For increased security the service should only listen on localhost
# Otherwise it could be reachable from the outside world without going through connect
task "whoami" {
driver = "docker"
config {
image = "containous/whoami:v1.5.0"
}
}
}
}
I hope that it makes thing clearer
You can use as many entrypoints and services as you want. But traefik is just one application, it will only use one certificate (which is the one from providers.consulcatalog.servicename
and also the one you will have to use in intentions).
I use my traefik service like this (for two entrypoints):
service {
name = "traefik"
port = "port1"
tags = ["port1"]
connect {
native = true
}
}
service {
name = "traefik"
port = "port2"
tags = ["port2"]
}
Note that one service can expose multiple ports (via tags) and one (!) connect {}
stanza is enough.
You can use as many entrypoints and services as you want. But traefik is just one application, it will only use one certificate (which is the one from
providers.consulcatalog.servicename
and also the one you will have to use in intentions).
Note that one service can expose multiple ports (via tags) and one (!)connect {}
stanza is enough.
Thanks a lot ! That's something I didn't know about the service exposure.
Here is a working example thanks to your input:
With proper intentions to allow ingress-demo
to speak to whoami1
and whoami2
, it works like a charm.
job "ingress-connect-demo" {
datacenters = ["meow"]
group "edge" {
network {
mode = "bridge"
port "http1" {
static = 8901
to = 8901
}
port "http2" {
static = 8902
to = 8902
}
# Treafik dashboard
port "dash" {
static = 8909
to = 8909
}
}
service {
name = "ingress-demo"
port = "http1"
tags = ["http1"]
connect {
native = true
}
}
service {
name = "ingress-demo"
port = "http2"
tags = ["http2"]
# connect {
# native = true
# }
}
task "traefik" {
driver = "docker"
config {
image = "traefik:2.6"
args = [
# Enables connect support, otherwise only http connections would be tried
"--providers.consulcatalog.connectaware=true",
# Make the communication secure by default
"--providers.consulcatalog.connectbydefault=true",
"--providers.consulcatalog.exposedbydefault=false",
"--entrypoints.http1=true",
"--entrypoints.http1.address=:8901",
"--entrypoints.http2=true",
"--entrypoints.http2.address=:8902",
"--entrypoints.traefik=true",
"--entrypoints.traefik.address=:8909",
"--api.dashboard=true",
"--api.insecure=true",
# The service name below should match the nomad/consul service above
# and is used for intentions in consul
"--providers.consulcatalog.servicename=ingress-demo",
"--providers.consulcatalog.prefix=ingress-demo",
# Automatically configured by Nomad through CONSUL_* environment variables
# as long as client consul.share_ssl is enabled
# "--providers.consulcatalog.endpoint.address=<socket|address>"
# "--providers.consulcatalog.endpoint.tls.ca=<path>"
# "--providers.consulcatalog.endpoint.tls.cert=<path>"
# "--providers.consulcatalog.endpoint.tls.key=<path>"
# "--providers.consulcatalog.endpoint.token=<token>"
]
}
env {
# Enable this if nomad is older than 1.1.3
# CONSUL_TLS_SERVER_NAME = "localhost"
}
}
}
group "backend1" {
network {
mode = "bridge"
}
service {
name = "whoami1"
port = 80
tags = [
"ingress-demo.enable=true",
"ingress-demo.http.routers.whoami1.rule=Host(`whoami1.example.com`)",
"ingress-demo.http.routers.whoami1.entrypoints=http1"
]
connect {
sidecar_service {}
}
}
# Note: For increased security the service should only listen on localhost
# Otherwise it could be reachable from the outside world without going through connect
task "whoami" {
driver = "docker"
config {
image = "containous/whoami:v1.5.0"
}
}
}
group "backend2" {
network {
mode = "bridge"
}
service {
name = "whoami2"
port = 80
tags = [
"ingress-demo.enable=true",
"ingress-demo.http.routers.whoami2.rule=Host(`whoami2.example.com`)",
"ingress-demo.http.routers.whoami2.entrypoints=http2"
]
connect {
sidecar_service {}
}
}
# Note: For increased security the service should only listen on localhost
# Otherwise it could be reachable from the outside world without going through connect
task "whoami" {
driver = "docker"
config {
image = "containous/whoami:v1.5.0"
}
}
}
}
And curl is happy:
$ curl -H "Host: whoami2.example.com" http://whoami2.example.com:8902/health/ --resolve whoami2.example.com:8902:10.xxx.xxx.12
Hostname: 0aa882d6ab30
IP: 127.0.0.1
IP: 172.26.74.189
RemoteAddr: 127.0.0.1:59626
GET /health/ HTTP/1.1
Host: whoami2.example.com
User-Agent: curl/7.81.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: xxx.xxx.xxx.xxx
X-Forwarded-Host: whoami2.example.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: 27c5f7da9048
X-Real-Ip: xxx.xxx.xxx.xxx
$ curl -H "Host: whoami1.example.com" http://whoami1.example.com:8901/health/ --resolve whoami1.example.com:8901:10.xxx.xxx.12
Hostname: ab3616414ad0
IP: 127.0.0.1
IP: 172.26.74.188
RemoteAddr: 127.0.0.1:59644
GET /health/ HTTP/1.1
Host: whoami1.example.com
User-Agent: curl/7.81.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: xxx.xxx.xxx.xxx
X-Forwarded-Host: whoami1.example.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: 27c5f7da9048
X-Real-Ip: xxx.xxx.xxx.xxx
Thanks for your help !
The sidecar is necessary for traefik?
Hi !
Thanks for that example, works like a charm!
I've one interrogation about the
providers.consulcatalog.servicename
that should match the nomad service name in theservice
stanza.It seems that this requirement makes it impossible to route multiple service via that traefik instance, am I right ?
The goal would be to use traefik deployed as a system job on each nomad client, and being able to serve as ingress-gateway for multiple services inside the service mesh.
That might be a limitation of the Traefik integration with consul-connect but I could not find proper explanation/documentation for this.
rgd,