flomotlik/gist:6212725

## gistfile1.md

      
    Raw
  

              gistfile1.md
            
          
    We have changed our check_url implementation from curl to wget for three reasons

It gives us the ability to retry on connection refused. Compare the command we used previously with curl:

curl -sSfL --retry 3 URL


to the current one:
wget --retry-connrefused --no-check-certificate -T 60 URL

The retry-connrefused triggers a retry whenever the connection is closed after it was already established to the server. This behaviour is for example seen when applications on Heroku take too long to boot and Heroku terminates the request. Then the connection is simply closed.
From the wget manpage:
--retry-connrefused
           Consider "connection refused" a transient error and try again.  Normally Wget gives up on a URL when it is unable to connect to the site because failure to connect is taken as a sign that the server is not running at all
           and that retries would not help.  This option is for mirroring unreliable sites whose servers tend to disappear for short periods of time.

You can try it yourself by running nc -l 5000 and then curl -sSfL --retry 3 localhost:5000. Now kill the nc process and see how curl dies, but wget --retry-connrefused --no-check-certificate -T 60.
Curls connect-timeout is not helpful here, as it specifically only works until the connection is established which isn't and has never been the problem for us.


Curl retries on HTTP Status 500 or other transient errors, but doesn't retry on connection refused as wget does. We do prefer the error to happen on connection refused. We have implemented a wrapper around wget that retries up to 3 times if it fails for any 500 Status.


Wget in its default configuration provides the more helpful output during a build. It shows when the connection is established and what is currently happening. Of course something we could probably achieve with curl as well, but I like to use what is already available.