Skip to content

Instantly share code, notes, and snippets.

@Tugzrida
Last active May 31, 2024 06:56
Show Gist options
  • Save Tugzrida/61235545dfc122262c69b0ab50265582 to your computer and use it in GitHub Desktop.
Save Tugzrida/61235545dfc122262c69b0ab50265582 to your computer and use it in GitHub Desktop.
MTA-STS Cloudflare worker
// This worker is designed to be able to neatly handle MTA-STS policies for multiple domains.
// Make a new worker with this script and add your domains to the stsPolicies dict like the example.
// Add a DNS AAAA record for mta-sts.yourdomain.com pointing to 100:: and set to proxied,
// then add a workers route for mta-sts.yourdomain.com/* pointing to this worker.
// You should probably also create a Cloudflare configuration rule disabling Browser Integrity Check for the mta-sts subdomain
// to ensure MTAs aren't blocked from retrieving your policy.
// You'll still need to manually add the appropriate _mta-sts.yourdomain.com TXT record to enable the policy,
// and the _smtp._tls.yourdomain.com TXT record for reporting.
const stsPolicies = {
"yourdomain.com":
`version: STSv1
mode: enforce
mx: mail.yourdomain.com
max_age: 86400`
}
const respHeaders = {
"Content-Type": "text/plain;charset=UTF-8",
"X-Clacks-Overhead": "GNU Terry Pratchett, Jon Postel, Alan Turing, Dan Kaminsky"
}
addEventListener("fetch", event => {
event.respondWith(handleRequest(event.request))
})
async function handleRequest(request) {
const reqUrl = new URL(request.url)
if (!reqUrl.hostname.startsWith("mta-sts.")) {
return new Response(`Incorrect worker route. mta-sts policies must be served on the mta-sts subdomain\n`, {status: 500, headers: respHeaders})
}
const policyHost = reqUrl.hostname.slice(8)
if (!stsPolicies.hasOwnProperty(policyHost)) {
return new Response(`${policyHost} is not defined in the mta-sts worker\n`, {status: 500, headers: respHeaders})
}
if (reqUrl.protocol !== "https:" || reqUrl.pathname !== "/.well-known/mta-sts.txt") {
reqUrl.protocol = "https:"
reqUrl.pathname = "/.well-known/mta-sts.txt"
return Response.redirect(reqUrl, 301)
}
return new Response(stsPolicies[policyHost] + "\n", {status: 200, headers: respHeaders})
}
@W4JEW
Copy link

W4JEW commented Apr 10, 2023

Hello! Thanks so much for sharing this!

Is it possible to specify multiple MX records in line 14 of this Worker? My email security provider requires me to add 3x MX records pointing to two domains. For example:

mydomain.com

MX  -  10  -  east.company.com
MX  -  10  -  west.company.com
MX  -  50  -  central.company.net

If all three had the same domain, I think I could use a wildcard (*):

mx: *.company.com

But I don't think I can do this:

mx: *.company.com
mx: central.company.net

Any thoughts?

@W4JEW
Copy link

W4JEW commented Apr 10, 2023

Actually, I think I just answered my own question! :-)

It looks like you can add each MX record on it's own line, and there's no need for a wildcard. So, in the case of my example, the following should work:

const stsPolicies = {
  "mydomain.com":
`version: STSv1
mode: enforce
mx: east.company.com
mx: west.company.com
mx: central.company.net
max_age: 86400`
}

I just set up something like that for one of my domains. I hope it works!

Thanks again!!!

@Tugzrida
Copy link
Author

@W4JEW The standard for MTA-STS policies is detailed in section 3.2 of RFC 8461.

It states that both multiple mx keys and simple leading wildcards are supported.

@W4JEW
Copy link

W4JEW commented Apr 10, 2023

Thanks for pointing me to that document. I got the Worker up and running - thanks to you! Great idea!

@BourbonCrow
Copy link

IT works great but, how would i add more domains then 1 to this? no clue about JS x,x i got it working fine for 1 domain

@W4JEW
Copy link

W4JEW commented Jan 29, 2024

Hello! Take a look at this post:

Hosting MTA-STS .txt file on CloudFlare Workers

@BourbonCrow
Copy link

BourbonCrow commented Jan 29, 2024

Hello! Take a look at this post:

Hosting MTA-STS .txt file on CloudFlare Workers

Thanks for quick reply. But i also manged to figure it out my self.
This one you posted works really well if all domains uses same mx records, dont even need to edit the code, i might use this one cause thats the case for me

@W4JEW
Copy link

W4JEW commented Jan 29, 2024

Excellent - was it the same method as described on the page I provided the link for?

@BourbonCrow
Copy link

BourbonCrow commented Jan 29, 2024

Excellent - was it the same method as described on the page I provided the link for?

yes i used it exactly like it was on that page and it works GREAT! its this one below here. and its a global one.. setting same MX records for all domains you use this worker for

const stsPolicies =
`version: STSv1
mode: enforce
mx: mail.protonmail.ch
mx: mailsec.protonmail.ch
max_age: 86400`

async function handleRequest(request) {
  return new Response(stsPolicies, {
    headers: {
      "content-type": "text/plain;charset=UTF-8",
    },
  })
}

addEventListener("fetch", event => {
  return event.respondWith(handleRequest(event.request))
})

This code below here is a modified version of original code to have global MX records if you use for example proton mail where all personal domains have the exact same MX records

const stsPolicies =
`version: STSv1
mode: enforce
mx: mail.protonmail.ch
mx: mailsec.protonmail.ch
max_age: 86400`

const respHeaders = {
  "Content-Type": "text/plain;charset=UTF-8"
}

addEventListener("fetch", event => {
  event.respondWith(handleRequest(event.request))
})

async function handleRequest(request) {
  const reqUrl = new URL(request.url)

  if (!reqUrl.hostname.startsWith("mta-sts.")) {
    return new Response(`Incorrect worker route. mta-sts policies must be served on the mta-sts subdomain\n`, {status: 500, headers: respHeaders})
  }

  if (reqUrl.protocol !== "https:" || reqUrl.pathname !== "/.well-known/mta-sts.txt") {
    reqUrl.protocol = "https:"
    reqUrl.pathname = "/.well-known/mta-sts.txt"
    return Response.redirect(reqUrl, 301)
  }

  return new Response(stsPolicies + "\n", {status: 200, headers: respHeaders})
}

and in original code if ppl are JS illiterates like me this is how you add more then 1 domain incase your different domains require different settings

const stsPolicies = {
  "yourdomain1.com":
`version: STSv1
mode: enforce
mx: mail.yourdomain1.com
mx: mailsec.yourdomain1.com
max_age: 86400`,
  "yourdomain2.com":
`version: STSv1
mode: enforce
mx: mail.yourdomain2.com
max_age: 86400`
}

@W4JEW
Copy link

W4JEW commented Feb 7, 2024

Could you share the contents of your Worker all in one code block instead of splitting it up across three? It's difficult to follow what the complete Worker should look like.

I've tried to emulate what's in the Gist, as well as based on your example, and whenever I go to verify the MTA-STS configuration, I get an error that the version (STSv1) cannot be detected.

Thank you!

FYI - mine currently looks like this:

const stsPolicies = {
  "domain1.com":
`version: STSv1
mode: testing
mx: aspmx.l.google.com
mx: alt1.aspmx.l.google.com
mx: alt2.aspmx.l.google.com
mx: aspmx2.googlemail.com
mx: aspmx3.googlemail.com
max_age: 604800`,
  "domain2.com":
`version: STSv1
mode: testing
mx: aspmx.l.google.com
mx: alt1.aspmx.l.google.com
mx: alt2.aspmx.l.google.com
mx: aspmx2.googlemail.com
mx: aspmx3.googlemail.com
max_age: 604800`,
  "domain3.com":
`version: STSv1
mode: testing
mx: aspmx.l.google.com
mx: alt1.aspmx.l.google.com
mx: alt2.aspmx.l.google.com
mx: aspmx2.googlemail.com
mx: aspmx3.googlemail.com
max_age: 604800`
}

const respHeaders = {
  "Content-Type": "text/plain;charset=UTF-8"
}

addEventListener("fetch", event => {
  event.respondWith(handleRequest(event.request))
})

async function handleRequest(request) {
  const reqUrl = new URL(request.url)

  if (!reqUrl.hostname.startsWith("mta-sts.")) {
    return new Response(`Incorrect worker route. mta-sts policies must be served on the mta-sts subdomain\n`, {status: 500, headers: respHeaders})
  }

  if (reqUrl.protocol !== "https:" || reqUrl.pathname !== "/.well-known/mta-sts.txt") {
    reqUrl.protocol = "https:"
    reqUrl.pathname = "/.well-known/mta-sts.txt"
    return Response.redirect(reqUrl, 301)
  }

  return new Response(stsPolicies + "\n", {status: 200, headers: respHeaders})
}

@BourbonCrow
Copy link

BourbonCrow commented Feb 7, 2024

@W4JEW no no you missunderstood me.. that is 2 different methods to do it.. the top 2 the first one is the one you sent me from the site.. the 2nd one is a modified version of original code. the bottom one is just explaining how to add more then 1 domain to the original code. my 2 are non domain specefic for example if all your MX records are the same for each domain you can use my code without having to add MX records for each domain individually

@W4JEW
Copy link

W4JEW commented Feb 7, 2024

I have unique MX records for several of my domains - so I have to explicitly define each domain.

So, out of your three code blocks, you're saying you can use either #1 or #2, then add multiple domains based on the example in #3?

@BourbonCrow
Copy link

BourbonCrow commented Feb 7, 2024

@W4JEW ok so if you have unique MX records for each domain .. you do it like this. if you need any help setting it up on cloudflare just ask and ill explain it

// This worker is designed to be able to neatly handle MTA-STS policies for multiple domains.

// Make a new worker with this script and add your domains to the stsPolicies dict like the example.
// Add a DNS AAAA record for mta-sts.yourdomain.com pointing to 100:: and set to proxied,
// then add a workers route for mta-sts.yourdomain.com/* pointing to this worker.

// You'll still need to manually add the appropriate _mta-sts.yourdomain.com TXT record to enable the policy, 
// and the _smtp._tls.yourdomain.com TXT record for reporting.

const stsPolicies = {
  "yourdomain1.com":
`version: STSv1
mode: enforce
mx: mail.yourdomain1.com
mx: mailsec.yourdomain1.com
max_age: 86400`,
  "yourdomain2.com":
`version: STSv1
mode: enforce
mx: mail.yourdomain2.com
max_age: 86400`,
  "yourdomain3.com":
`version: STSv1
mode: enforce
mx: mail.yourdomain3.com
mx: mailsec.yourdomain3.com
max_age: 86400`
}


const respHeaders = {
  "Content-Type": "text/plain;charset=UTF-8"
}

addEventListener("fetch", event => {
  event.respondWith(handleRequest(event.request))
})

async function handleRequest(request) {
  const reqUrl = new URL(request.url)

  if (!reqUrl.hostname.startsWith("mta-sts.")) {
    return new Response(`Incorrect worker route. mta-sts policies must be served on the mta-sts subdomain\n`, {status: 500, headers: respHeaders})
  }

  const policyHost = reqUrl.hostname.slice(8)

  if (!stsPolicies.hasOwnProperty(policyHost)) {
    return new Response(`${policyHost} is not defined in the mta-sts worker\n`, {status: 500, headers: respHeaders})
  }

  if (reqUrl.protocol !== "https:" || reqUrl.pathname !== "/.well-known/mta-sts.txt") {
    reqUrl.protocol = "https:"
    reqUrl.pathname = "/.well-known/mta-sts.txt"
    return Response.redirect(reqUrl, 301)
  }

  return new Response(stsPolicies[policyHost] + "\n", {status: 200, headers: respHeaders})
}

@Ry3nlNaToR
Copy link

Ry3nlNaToR commented May 30, 2024

A warning cloudflare security features might causes issues, when using wokers CF protection is still enabled.

A friend was using a worker to host his mta-sts policy was alerted to a issue when a family member of him e-mailed him and it bounced the error mentioned MTA-STS validation error.

He investigated policy looked fine after looking on Cloudflare dash, security event the sending mail server requests for the policy file got served with a challenge page instead.

To minimise the potential chances of this happening again. he created a rule for request to MTA-STS hostname set security level to essentially off and disabled browser integrity check.

@W4JEW
Copy link

W4JEW commented May 31, 2024

@Ry3nlNaToR could you be more specific about what happened here? I'm not sure what you mean by "workers (spelling?) CF protection is still enabled".

This sounds like something self-imposed and likely due to a misconfiguration in Cloudflare's WAF for the zone the MTA-STS Worker is associated with.

There are no inherent policies imposed on Workers that would force an Interactive/Managed/JavaScript challenge. The only way this would occur would be for the administrator to have a layer 7 firewall rule (WAF Custom Rule) matching on the URL (Worker binding) that invokes a challenge. That would be something that the administrator would have to add - and it would be very bad practice to have it enabled on the specific binding for the MTA-STS Worker.

I have a WAF Custom Rule that matches on the MTA-STS hostname and is set to skip as follows:

If hostname = mta-sts.mydomain.com
Action = Skip

The following are skipped: All remaining custom rules, all rate limiting rules, all managed rules, all Super Bot Fight Mode Rules, Zone Lockdown, Browser Integrity Check, Hotlink Protection, and Security Level.

Basically - anything can access the MTA-STS Worker and will never get presented with a challenge of any sort.

If you can get more information on what happened, that would be very helpful. And apologies to @Tugzrida for this! Hopefully it's telling you how helpful your contribution has been! :-)

@Tugzrida
Copy link
Author

I would imagine it's most likely Cloudflare's Browser Integrity Check, which from memory is enabled by default and blocks requests from some User-Agents. The main one I run into is Python urllib's default UA, but it's possible CF has some MTA UAs in their naughty list. I'll add a note to the gist

@W4JEW
Copy link

W4JEW commented May 31, 2024

With all due respect, Browser Integrity Check is not solely responsible for issuing a challenge. BIC is a mechanism in which Cloudflare fingerprints a browser to determine whether it's a legitimate browser or a headless device.

BIC is loosely based Google's Picasso - not to be confused with Picasa - Google's photo editor.

BIC is indeed taken into consideration when it comes to determining if a device is suspect - but there's a lot more that needs to happen before Cloudflare would take any action (automated). I can all but guarantee this is manually invoked by a WAF Custom Rule...especially if this is a Free/Pro/Biz customer. I would suspect something like Bot Management could be a culprit - but BM is a very expensive product and anyone employing it would know to create exclusions for automated checks (i.e. APIs - or automated checks like quering mta-sta.mydomain.com for a mail policy).

If you call out anything in your Gist - I would suggest adding something like this:

Ensure your Cloudflare security policy does not invoke a challenge for any requests to the URL associated with the Worker binding.

But it might be good to get a response back from @Ry3nlNaToR first to see if they can provide more insight into what actually happened.

@Tugzrida
Copy link
Author

The Cloudflare docs are quite clear that this is exactly the behaviour of BIC:

Cloudflare’s Browser Integrity Check (BIC) looks for common HTTP headers abused most commonly by spammers and denies access to your page.

It also challenges visitors without a user agent or with a non-standard user agent such as commonly used by abusive bots, crawlers, or visitors.

I can confirm that on one of my domains where I use this worker, on which I don't have any non-default rules configured, simply setting the User-Agent header to Python-urllib/3.11 is enough to get BIC to block the request.

@W4JEW
Copy link

W4JEW commented May 31, 2024

Okie dokie - I will step away from the conversation... It's your post, anyway. Thanks so much for sharing the information in the first place! It was IMMENSELY helpful!

@Ry3nlNaToR
Copy link

@Tugzrida @W4JEW Its not my CF account I host my policy on AWS using CloudFront and S3, anyway back the subject I asked my friend for more details wasn't a managed challenge it was a block as suspected by @Tugzrida it was due to BIC Service: Browser integrity check Action taken:Block the user agent was libwww-perl/6.68 disabling Browser Integrity Check would fix it or just do what you did WAF Custom Rule and skip all of the security features.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment