route53 hook for dehydrated - python2 / python3 + boto2 version. Tested on Ubuntu 16.04

route53.py

#!/usr/bin/env python

# How to use:
#
# Ubuntu 16.04: apt install -y python-boto OR apt install -y python3-boto
#
# Specify the default profile on aws/boto profile files or use the optional AWS_PROFILE env var:
# AWS_PROFILE=example ./dehydrated -c -d example.com -t dns-01 -k /etc/dehydrated/hooks/route53.py
#
# Manually specify hosted zone:
# HOSTED_ZONE=example.com AWS_PROFILE=example ./dehydrated -c -d example.com -t dns-01 -k /etc/dehydrated/hooks/route53.py
#
# More info about dehaydrated and dns challenge: https://github.com/lukas2511/dehydrated/wiki/Examples-for-DNS-01-hooks
# Using AWS Profiles: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-multiple-profiles

import os
import sys
from boto.route53 import *
from time import sleep


def route53_dns(domain, txt_challenge, action='upsert'):

    conn = connection.Route53Connection()
    action = action.upper()

    if 'HOSTED_ZONE' in os.environ:
      hosted_zone = os.environ['HOSTED_ZONE']
      if not domain.endswith(hosted_zone):
        raise Exception("Incorrect hosted zone for domain {0}".format(domain))
      zone = conn.get_hosted_zone_by_name("{0}.".format(hosted_zone))
      zone_id = zone['GetHostedZoneResponse']['HostedZone']['Id'].replace('/hostedzone/', '')
    else:
      zones = conn.get_all_hosted_zones()
      for zone in zones['ListHostedZonesResponse']['HostedZones']:
        if "{0}.".format(domain).endswith(zone['Name']):
          zone_id = zone['Id'].replace('/hostedzone/', '')
          break
      else:
        raise Exception("Hosted zone not found for domain {0}".format(domain))

    name = u'_acme-challenge.{0}.'.format(domain)
    record_set = conn.get_all_rrsets(zone_id, name=name, type='TXT')
    challenges = [u'"{0}"'.format(txt_challenge)]

    for r in record_set:
        if r.name == name and r.type == 'TXT':
            challenges += r.resource_records

    change_set = record.ResourceRecordSets(conn, zone_id)
    change = change_set.add_change("{0}".format(action), '_acme-challenge.{0}'.format(domain), type='TXT', ttl=60)
    for c in set(challenges):
      change.add_value(c)

    try:
      response = change_set.commit()
    except Exception as e:
      if action == "DELETE":
        pass
      else:
        print(e.message, e.args)

    if action in ('CREATE', 'UPSERT'):
      # wait for DNS update
      timeout = 300
      sleep_time = 5
      time_elapsed = 0
      st = status.Status(conn, response['ChangeResourceRecordSetsResponse']['ChangeInfo'])
      while st.update() != 'INSYNC' and time_elapsed <= timeout:
          print("Waiting for DNS change to complete... (Elapsed {0} seconds)".format(time_elapsed))
          sleep(sleep_time)
          time_elapsed += sleep_time

      if st.update() != 'INSYNC' and time_elapsed > timeout:
          raise Exception("Timed out while waiting for DNS record to be ready. Waited {0} seconds".format(time_elapsed))

      print("DNS change completed")

if __name__ == "__main__":
    hook = sys.argv[1]
    if len(sys.argv) > 2:
      domain = sys.argv[2]
      txt_challenge = sys.argv[4]
    else:
      domain = None
      txt_challenge = None

    if hook == "deploy_challenge":
        action = 'upsert'
    elif hook == "clean_challenge":
        action = 'delete'
    else:
        sys.exit(0)

    print("hook: {0}".format(hook))
    print("domain: {0}".format(domain))
    print("txt_challenge: {0}".format(txt_challenge))

    route53_dns(domain, txt_challenge, action)

@Tea23 @rmarchei

I tried to reproduce the error you mentioned but I wasn't able to. I created a certificate only for *.example.com and it worked, then I added the example.com domain and it still worked.

Letsencrypt has cached those challenges, so I'll wait a bit and try generating *.example.com and example.com at the same time. Also note that I'm using the staging environment.

Do you still have issues with the script or have you been able to find a workaround? Could you give us more information, like the exact error message with sensitive information cesored?

@mirath your changes break the script when HOSTED_ZONE is set, the ident on line change_set = record.ResourceRecordSets(conn, zone_id) is wrong, it should not be under if

@gytisgreitai you are right. I'll fix it, though if you have a patch feel free to upload it to the gist

@mirath

This is attempting to produce a cert for *.scoutlink.net and scoutlink.net.

Ignoring startup_hook
Processing *.scoutlink.net with alternative names: scoutlink.net 
 + Checking domain name(s) of existing cert... unchanged.
 + Checking expire date of existing cert...
 + Valid till Jun 15 23:55:16 2018 GMT (Less than 30 days). Renewing!
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 2 authorizations URLs from the CA
 + Handling authorization for scoutlink.net
 + Handling authorization for scoutlink.net
 + 2 pending challenge(s)
 + Deploying challenge tokens...
hook: deploy_challenge
domain: scoutlink.net
txt_challenge: 047GnpkfdgScW4-u7_zljXXOxs1snEUDrPAMzyiBy-Q
Waiting for DNS change to complete... (Elapsed 0 seconds)
Waiting for DNS change to complete... (Elapsed 5 seconds)
Waiting for DNS change to complete... (Elapsed 10 seconds)
Waiting for DNS change to complete... (Elapsed 15 seconds)
Waiting for DNS change to complete... (Elapsed 20 seconds)
Waiting for DNS change to complete... (Elapsed 25 seconds)
Waiting for DNS change to complete... (Elapsed 30 seconds)
Waiting for DNS change to complete... (Elapsed 35 seconds)
Waiting for DNS change to complete... (Elapsed 40 seconds)
DNS change completed
hook: deploy_challenge
domain: scoutlink.net
txt_challenge: YaqTDtzLjFKncHxrfFAI1iJrlyt7svFZIwkaD3jhl0g
Waiting for DNS change to complete... (Elapsed 0 seconds)
Waiting for DNS change to complete... (Elapsed 5 seconds)
Waiting for DNS change to complete... (Elapsed 10 seconds)
Waiting for DNS change to complete... (Elapsed 15 seconds)
Waiting for DNS change to complete... (Elapsed 20 seconds)
Waiting for DNS change to complete... (Elapsed 25 seconds)
Waiting for DNS change to complete... (Elapsed 30 seconds)
Waiting for DNS change to complete... (Elapsed 35 seconds)
DNS change completed
 + Responding to challenge for scoutlink.net authorization...
hook: clean_challenge
domain: scoutlink.net
txt_challenge: 047GnpkfdgScW4-u7_zljXXOxs1snEUDrPAMzyiBy-Q
Traceback (most recent call last):
  File "/etc/dehydrated/hook.py", line 88, in <module>
    route53_dns(domain, txt_challenge, action)
  File "/etc/dehydrated/hook.py", line 38, in route53_dns
    response = change_set.commit()
  File "/usr/lib/python2.7/dist-packages/boto/route53/record.py", line 168, in commit
    return self.connection.change_rrsets(self.hosted_zone_id, self.to_xml())
  File "/usr/lib/python2.7/dist-packages/boto/route53/connection.py", line 475, in change_rrsets
    body)
boto.route53.exception.DNSServerError: DNSServerError: 400 Bad Request
<?xml version="1.0"?>
<ErrorResponse xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><Error><Type>Sender</Type><Code>InvalidChangeBatch</Code><Message>Tried to delete resource record set [name='_acme-challenge.scoutlink.net.', type='TXT'] but the values provided do not match the current values</Message></Error><RequestId>e56f9833-5b74-11e8-bd24-0713c2fe4f99</RequestId></ErrorResponse>

The indentation in the above code seems to be off, this worked for me:

#!/usr/bin/env python

# How to use:
#
# Ubuntu 16.04: apt install -y python-boto OR apt install -y python3-boto
#
# Specify the default profile on aws/boto profile files or use the optional AWS_PROFILE env var:
# AWS_PROFILE=example ./dehydrated -c -d example.com -t dns-01 -k /etc/dehydrated/hooks/route53.py
#
# Manually specify hosted zone:
# HOSTED_ZONE=example.com AWS_PROFILE=example ./dehydrated -c -d example.com -t dns-01 -k /etc/dehydrated/hooks/route53.py
#
# More info about dehaydrated and dns challenge: https://github.com/lukas2511/dehydrated/wiki/Examples-for-DNS-01-hooks
# Using AWS Profiles: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-multiple-profiles

import os
import sys
from boto.route53 import *
from time import sleep


def route53_dns(domain, txt_challenge, action='upsert'):

    conn = connection.Route53Connection()

    if 'HOSTED_ZONE' in os.environ:
        hosted_zone = os.environ['HOSTED_ZONE']
        if not domain.endswith(hosted_zone):
            raise Exception("Incorrect hosted zone for domain {0}".format(domain))
        zone = conn.get_hosted_zone_by_name("{0}.".format(hosted_zone))
        zone_id = zone['GetHostedZoneResponse']['HostedZone']['Id'].replace('/hostedzone/', '')
    else:
        zones = conn.get_all_hosted_zones()
        candidate_zones = []
        domain_dot = "{0}.".format(domain)
        for zone in zones['ListHostedZonesResponse']['HostedZones']:
            if domain_dot.endswith(zone['Name']):
                candidate_zones.append((domain_dot.find(zone['Name']), zone['Id'].replace('/hostedzone/', '')))

        if len(candidate_zones) == 0:
            raise Exception("Hosted zone not found for domain {0}".format(domain))

        candidate_zones.sort()
        zone_id = candidate_zones[0][1]

    change_set = record.ResourceRecordSets(conn, zone_id)
    change = change_set.add_change("{0}".format(action.upper()), '_acme-challenge.{0}'.format(domain), type='TXT', ttl=60)
    change.add_value('"{0}"'.format(txt_challenge))
    response = change_set.commit()

    if action.upper() == 'UPSERT':
        # wait for DNS update
        timeout = 300
        sleep_time = 5
        time_elapsed = 0
        st = status.Status(conn, response['ChangeResourceRecordSetsResponse']['ChangeInfo'])
        while st.update() != 'INSYNC' and time_elapsed <= timeout:
            print("Waiting for DNS change to complete... (Elapsed {0} seconds)".format(time_elapsed))
            sleep(sleep_time)
            time_elapsed += sleep_time

        if st.update() != 'INSYNC' and time_elapsed > timeout:
            raise Exception("Timed out while waiting for DNS record to be ready. Waited {0} seconds".format(time_elapsed))

        print("DNS change completed")

if __name__ == "__main__":
    hook = sys.argv[1]

    if hook == "deploy_challenge":
        domain = sys.argv[2]
        txt_challenge = sys.argv[4]
        action = 'upsert'
    elif hook == "clean_challenge":
        domain = sys.argv[2]
        txt_challenge = sys.argv[4]
        action = 'delete'
    elif hook == "startup_hook":
        print("Ignoring startup_hook")
        exit(0)
    elif hook == "exit_hook":
        print("Ignoring exit_hook")
        exit(0)
    elif hook == "deploy_cert":
        print("Ignoring deploy_cert hook")
        exit(0)
    elif hook == "unchanged_cert":
        print("Ignoring unchanged_cert hook")
        exit(0)
    else:
        print("Ignoring unknown hook %s", hook)
        exit(0)

    print("hook: {0}".format(hook))
    print("domain: {0}".format(domain))
    print("txt_challenge: {0}".format(txt_challenge))

    route53_dns(domain, txt_challenge, action)```

@niall-byrne Thanks!!!

Is any of the versions above able to use wildcards? I'm still getting the error but the values provided do not match the current values as mentioned above (https://gist.github.com/rmarchei/98489c05f0898abe612eec916508f2bf#gistcomment-2594451).

I've written a fork of this gist to:

• Implement wildcard support
• Allow HOOK_CHAIN="yes" for much faster batch confirmation (only need to wait for DNS propagation once, not once per domain)

https://gist.github.com/nh2/f744ac591e95f0c25b501db00cf7c71a