Skip to content

Instantly share code, notes, and snippets.

@minaguib

minaguib/sectigo.md

Last active November 5, 2020 11:12
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save minaguib/c8db186af450bceaaa7c452b76a9901b to your computer and use it in GitHub Desktop.
Save minaguib/c8db186af450bceaaa7c452b76a9901b to your computer and use it in GitHub Desktop.
Download ZIP
Sectigo/Comodo/USERTrust/AddTrust Certificate Expiry May 30 2020 notes
Raw
sectigo.md

Comodo/Sectigo/USERTrust/AddTrust root certificate expiry

On May 30th 2020 (10:48am GMT / 6:48am EDT / 3:48am PDT), several root & intermediate certificates that were part of the Comodo family expired.

The expired certificates

The following certificates expired on May 30 10:48:38 2020 GMT:

Impact

Thousands of web sites / services / APIs which present certificates that were issued by the vendor may have experienced trouble negotiating incoming secure connections from their clients. Exact issues, if any, depended on a mix of server configuration, client version, and client configuration.

A client validating any of those certificates had access to 3 "paths" to do so successfully. One of those paths (A) became invalid after the above certificates expired:

Trust Chain Path A: (INVALID)

Trust Chain Path B: (VALID)

  • USERTrust RSA Certification Authority (Type:Root) (Serial:01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d)
  • Sectigo RSA DV/OV/EV Secure Server CA (Type:Intermediate)
  • End Entity (Type:Leaf Certificate)

Trust Chain Path C: (VALID)

Problems and solutions

Client reports certificate not issued by trusted Certificate Authority

To address:

Client-side:

Ensure client updates their local CA roots trust store to include one/both of the (Type:Root) certificates in either of the Path B or Path C cases above

Server-side:

In the chain bundle you serve, include the (Type:Intermediate) certificates in path B and/or C above to encourage your clients to reach one of the two valid (Type:Root) certificates in their local CA roots trust store.

OpenSSL-based clients reporting certificate is expired

If a server is sending a bundled chain of certificates and it includes the above NOW EXPIRED certificates from Chain Path A, AND the client is using an older OpenSSL version (<1.1.0), AND the client's local CA roots trust store has the "AddTrust" root certificate, it will trigger a bug where the client will report an Expired error.

To address, do any/some/most of the following:

Server-side:

Update the server to no longer include the above 2 expired certificates in the chain bundle.

Client-side:

Update the OpenSSL library software version.

Edit the local CA roots trust store and remove/disable/blacklist the "AddTrust External CA Root" root certificate.

More details

@minaguib
Copy link
Author

@asurak Thanks, done.

@minaguib
Copy link
Author

Also, TIL, client validation walking up the chain just looks for the issuer by CN. There can be multiple matches (from server-supplied bundle and/or local store), some valid some not... !!

@asurak
Copy link

asurak commented May 30, 2020

Great! The chain with AAA works also on old systems with old openssl, so it has the higher level of compatibility, if needed.

@kawesam
Copy link

kawesam commented May 31, 2020

How do I delete this two files on ubuntu 16 to resolve the issue? Update the server to no longer include the above 2 expired certificates in the chain bundle.

@minaguib
Copy link
Author

@kawesam I'm not sure about Ubuntu/16 specifics.

If you're running an HTTPS server and you think you're sending the expired intermediate/root in the chain, check with https://whatsmychaincert.com/ and if that's the case, it depends on the web server you're using, but it'll typically involve editing a PEM file and removing some certs from it.

If you're running a client on an Ubuntu 16 server, see if the instructions at the bottom of Andrew's blog post help

@kamilbednarz
Copy link

My band-aid for AWS Beanstalk, Amazon Linux (CentOS) & .ebextensions:

# On May 30, 2020, the AddTrust root certificate that is still used in some SSL cert authorization
# chains (e.g. our case: xxx.yyy.com). OpenSSL 1.0.x in Amazon Linux is not able to fall back
# from the expired resolution path to a valid one.
# Solution to this is to blacklist expired certificate, it won't be used for the cert validations.
 
# More details here:
# https://calnetweb.berkeley.edu/calnet-technologists/incommon-sectigo-certificate-service/addtrust-external-root-expiration-may-2020
 
files:
  /etc/pki/ca-trust/source/blacklist/add_trust.cer:
    mode: "000755"
    owner: root
    group: root
    content: |
      -----BEGIN TRUSTED CERTIFICATE-----
      MIIENjCCAx6gAwIBAgIBATANBgkqhkiG9w0BAQUFADBvMQswCQYDVQQGEwJTRTEU
      MBIGA1UEChMLQWRkVHJ1c3QgQUIxJjAkBgNVBAsTHUFkZFRydXN0IEV4dGVybmFs
      IFRUUCBOZXR3b3JrMSIwIAYDVQQDExlBZGRUcnVzdCBFeHRlcm5hbCBDQSBSb290
      MB4XDTAwMDUzMDEwNDgzOFoXDTIwMDUzMDEwNDgzOFowbzELMAkGA1UEBhMCU0Ux
      FDASBgNVBAoTC0FkZFRydXN0IEFCMSYwJAYDVQQLEx1BZGRUcnVzdCBFeHRlcm5h
      bCBUVFAgTmV0d29yazEiMCAGA1UEAxMZQWRkVHJ1c3QgRXh0ZXJuYWwgQ0EgUm9v
      dDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALf3GjPm8gAELTngTlvt
      H7xsD821+iO2zt6bETOXpClMfZOfvUq8k+0DGuOPz+VtUFrWlymUWoCwSXrbLpX9
      uMq/NzgtHj6RQa1wVsfwTz/oMp50ysiQVOnGXw94nZpAPA6sYapeFI+eh6FqUNzX
      mk6vBbOmcZSccbNQYArHE504B4YCqOmoaSYYkKtMsE8jqzpPhNjfzp/haW+710LX
      a0Tkx63ubUFfclpxCDezeWWkWaCUN/cALw3CknLa0Dhy2xSoRcRdKn23tNbE7qzN
      E0S3ySvdQwAl+mG5aWpYIxG3pzOPVnVZ9c0p10a3CitlttNCbxWyuHv77+ldU9U0
      WicCAwEAAaOB3DCB2TAdBgNVHQ4EFgQUrb2YejS0Jvf6xCZU7wO94CTLVBowCwYD
      VR0PBAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wgZkGA1UdIwSBkTCBjoAUrb2YejS0
      Jvf6xCZU7wO94CTLVBqhc6RxMG8xCzAJBgNVBAYTAlNFMRQwEgYDVQQKEwtBZGRU
      cnVzdCBBQjEmMCQGA1UECxMdQWRkVHJ1c3QgRXh0ZXJuYWwgVFRQIE5ldHdvcmsx
      IjAgBgNVBAMTGUFkZFRydXN0IEV4dGVybmFsIENBIFJvb3SCAQEwDQYJKoZIhvcN
      AQEFBQADggEBALCb4IUlwtYj4g+WBpKdQZic2YR5gdkeWxQHIzZlj7DYd7usQWxH
      YINRsPkyPef89iYTx4AWpb9a/IfPeHmJIZriTAcKhjW88t5RxNKWt9x+Tu5w/Rw5
      6wwCURQtjr0W4MHfRnXnJK3s9EK0hZNwEGe6nQY1ShjTK3rMUUKhemPR5ruhxSvC
      Nr4TDea9Y355e6cJDUCrat2PisP29owaQgVR1EX1n6diIWgVIEM8med8vSTYqZEX
      c4g/VhsxOBi0cQ+azcgOno4uG+GMmIPLHzHxREzGBHNJdmAPx/i9F4BrLunMTA5a
      mnkPIAou1Z5jJh5VkpTYghdae9C8x49OhgQwODAeBggrBgEFBQcDAwYIKwYBBQUH
      AwEGCCsGAQUFBwMEDBZBZGRUcnVzdCBFeHRlcm5hbCBSb290
      -----END TRUSTED CERTIFICATE-----
 
commands:
  01_enable_dynamic_cert_files_evaluation:
    command: "update-ca-trust enable"
  02_extract_certs:
    command: "update-ca-trust extract"

But for non-beanstalk servers: it's a matter of placing this cert in the blacklist directory & re-generating ca-trust new files.

@emmm-dee
Copy link

emmm-dee commented Jun 1, 2020

Well this made for an interesting start of the week :(

@minaguib
Copy link
Author

minaguib commented Jun 2, 2020

@ValZapod Thanks. I noted COMODO RSA Certification Authority in the "expired certificates" list.

I'm thinking now of ways to better represent the variations in INVALID and VALID chain paths...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment