Skip to content

Instantly share code, notes, and snippets.

@ehazlett
Last active March 1, 2016 14:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ehazlett/8df149fa1a4c9f069b79 to your computer and use it in GitHub Desktop.
Save ehazlett/8df149fa1a4c9f069b79 to your computer and use it in GitHub Desktop.

Current Interlock:

Set the desired port in the INTERLOCK_DATA env var: -e INTERLOCK_DATA='{"port":80}'

Interlock NG:

Set the desired port in a label: --label interlock.port=80

@twobombs
Copy link

I've implemented the change, we see that some containers are not picked @ port 80 but on another port. One site that continuisly has this has fixed port, 30001, 30002 and 30003. 30001 is published under port 80, yet Interlock keeps on picking port 30003, which is MYSQL ( needed to feed the MySQL server). We see similar behaviour in other dynamicly published port, but the ones with the fixed port are notorious for picking the wrong port.

log snippet:

works and publishes correct: ( chance ~ 1 in 10 )

"�2016-02-29T11:34:37.669957771Z time="2016-02-29T11:34:37Z" level="info" msg="[haproxy] []: upstream=:30001 container="

but when a container is added or removed the following pops up:

2016-02-29T12:15:05.495103290Z time="2016-02-29T12:15:05Z" level="info" msg="[haproxy] : upstream=:30003 container="

port 30001 is port 80, 30003 is MySQL and 30002 is ssh

We see similar behaviour when ports are dynamically allocated and with fewer ports publised ( eg: 80 & ssh )

This stops a lot of our plans we have for docker and is a blocker in that we can't add or remove containers without worrying that some ports are incorrectly published to the outside word ( eg: no website )

Would it be wise to file a bug report of this, or has this already been observed by others ?

@ehazlett
Copy link
Author

ehazlett commented Mar 1, 2016

This looks like it is working as expected:

The following gets port "80" exposed as it's the only port.

docker run --rm -ti -p 80:80 --hostname foo.local alpine ash

interlock:ehazlett interlock> ./cmd/interlock/interlock -D run -c ~/Sync/interlock.conf.haproxy                          
INFO[0000] interlock 1.0.0 (0f87cb0)                    
DEBU[0000] docker client: url=unix:///var/run/docker.sock 
DEBU[0000] loading extension: name=haproxy configpath=/tmp/haproxy.cfg 
DEBU[0000] starting event handling                      
DEBU[0000] checking to reload                           
DEBU[0000] reloading                                    
DEBU[0000] updating load balancers                      
DEBU[0000] generating proxy config                       ext=haproxy
INFO[0000] configuration updated                         ext=haproxy
DEBU[0000] dropping SYN packets to trigger client re-send  ext=haproxy
DEBU[0000] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0000] error signaling clients to resend; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0000] resuming SYN packets                          ext=haproxy
DEBU[0000] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0000] error signaling clients to resume; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0000] reload duration: 14.13ms                     
DEBU[0010] event received: type=create id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0010] event received: type=attach id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0010] event received: type= id=                    
DEBU[0010] event received: type=start id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0010] inspecting container: id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0010] checking container labels: id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0010] checking container ports: id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0010] container is monitored; triggering reload: id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0010] triggering reload                            
DEBU[0010] event received: type=resize id=701d1aedb4fc3777bc4e95815918ce0ec2b86ad867fe3f34ab5e033bcb946943 
DEBU[0011] reaping key: reload                          
DEBU[0011] triggering reload from cache                 
DEBU[0011] checking to reload                           
DEBU[0011] reloading                                    
DEBU[0011] updating load balancers                      
DEBU[0011] generating proxy config                       ext=haproxy
INFO[0011] foo.local: upstream=172.17.0.1:80 container=determined_wozniak  ext=haproxy
DEBU[0011] alias domains: []                             ext=haproxy
DEBU[0011] adding host name=foo_local domain=foo.local   ext=haproxy
INFO[0011] configuration updated                         ext=haproxy

If more than one port is published I can specify which port to use:

docker run --rm -ti -p 80:80 -p 3000:3000 --label interlock.port=80 --hostname foo.local alpine ash

You can see the upstream is 172.17.0.1:80

INFO[0000] interlock 1.0.0 (0f87cb0)                    
DEBU[0000] docker client: url=unix:///var/run/docker.sock 
DEBU[0000] loading extension: name=haproxy configpath=/tmp/haproxy.cfg 
DEBU[0000] starting event handling                      
DEBU[0000] checking to reload                           
DEBU[0000] reloading                                    
DEBU[0000] updating load balancers                      
DEBU[0000] generating proxy config                       ext=haproxy
INFO[0000] configuration updated                         ext=haproxy
DEBU[0000] dropping SYN packets to trigger client re-send  ext=haproxy
DEBU[0000] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0000] error signaling clients to resend; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0000] resuming SYN packets                          ext=haproxy
DEBU[0000] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0000] error signaling clients to resume; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0000] reload duration: 12.87ms                     
DEBU[0007] event received: type=create id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0007] event received: type=attach id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0007] event received: type= id=                    
DEBU[0007] event received: type=start id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0007] inspecting container: id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0007] checking container labels: id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0007] checking container ports: id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0007] container is monitored; triggering reload: id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0007] triggering reload                            
DEBU[0007] event received: type=resize id=07223ce67654856216f540cfcf5d1b08af79c9766f61671578d554d7e796cd5c 
DEBU[0008] reaping key: reload                          
DEBU[0008] triggering reload from cache                 
DEBU[0008] checking to reload                           
DEBU[0008] reloading                                    
DEBU[0008] updating load balancers                      
DEBU[0008] generating proxy config                       ext=haproxy
DEBU[0008] foo.local: found specified port 80 exposed as 80  ext=haproxy
INFO[0008] foo.local: upstream=172.17.0.1:80 container=admiring_ptolemy  ext=haproxy
DEBU[0008] alias domains: []                             ext=haproxy
DEBU[0008] adding host name=foo_local domain=foo.local   ext=haproxy
INFO[0008] configuration updated                         ext=haproxy
DEBU[0008] dropping SYN packets to trigger client re-send  ext=haproxy
DEBU[0008] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0008] error signaling clients to resend; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0008] resuming SYN packets                          ext=haproxy
DEBU[0008] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0008] error signaling clients to resume; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0008] reload duration: 13.90ms                     

I can change the port specification for interlock and it adheres:

docker run --rm -ti -p 80:80 -p 3000:3000 --label interlock.port=3000 --hostname foo.local alpine ash

Here I'm specifying port 3000 instead of 80.

INFO[0000] interlock 1.0.0 (0f87cb0)                    
DEBU[0000] docker client: url=unix:///var/run/docker.sock 
DEBU[0000] loading extension: name=haproxy configpath=/tmp/haproxy.cfg 
DEBU[0000] starting event handling                      
DEBU[0000] checking to reload                           
DEBU[0000] reloading                                    
DEBU[0000] updating load balancers                      
DEBU[0000] generating proxy config                       ext=haproxy
INFO[0000] configuration updated                         ext=haproxy
DEBU[0000] dropping SYN packets to trigger client re-send  ext=haproxy
DEBU[0000] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0000] error signaling clients to resend; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0000] resuming SYN packets                          ext=haproxy
DEBU[0000] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0000] error signaling clients to resume; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0000] reload duration: 14.54ms                     
DEBU[0004] event received: type=create id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0004] event received: type=attach id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0004] event received: type= id=                    
DEBU[0004] event received: type=start id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0004] inspecting container: id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0004] checking container labels: id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0004] checking container ports: id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0004] container is monitored; triggering reload: id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0004] triggering reload                            
DEBU[0004] event received: type=resize id=e6cbd64c80f8e5577662ee806d6a3a321040b1230478c836dc9b58feb6b9326c 
DEBU[0005] reaping key: reload                          
DEBU[0005] triggering reload from cache                 
DEBU[0005] checking to reload                           
DEBU[0005] reloading                                    
DEBU[0005] updating load balancers                      
DEBU[0005] generating proxy config                       ext=haproxy
DEBU[0005] foo.local: found specified port 3000 exposed as 3000  ext=haproxy
INFO[0005] foo.local: upstream=172.17.0.1:3000 container=sleepy_almeida  ext=haproxy
DEBU[0005] alias domains: []                             ext=haproxy
DEBU[0005] adding host name=foo_local domain=foo.local   ext=haproxy
INFO[0005] configuration updated                         ext=haproxy
DEBU[0005] dropping SYN packets to trigger client re-send  ext=haproxy
DEBU[0005] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0005] error signaling clients to resend; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0005] resuming SYN packets                          ext=haproxy
DEBU[0005] &{/sbin/iptables [/sbin/iptables -I INPUT -p tcp --dport 80 --syn -j DROP] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}  ext=haproxy
WARN[0005] error signaling clients to resume; you will notice dropped packets: exit status 3  ext=haproxy
DEBU[0005] reload duration: 15.45ms 

You can see the upstream selected is using port 3000.

@ehazlett
Copy link
Author

ehazlett commented Mar 1, 2016

FWIW I also verified this with the nginx extension as well. Both working as expected. If you continue to have issues please file an issue on github. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment