Skip to content

Instantly share code, notes, and snippets.

@lzjluzijie
Last active July 28, 2023 04:36
Show Gist options
  • Star 23 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save lzjluzijie/3f1294f47755972f9bcd68f90ffc0b73 to your computer and use it in GitHub Desktop.
Save lzjluzijie/3f1294f47755972f9bcd68f90ffc0b73 to your computer and use it in GitHub Desktop.
Access Private BackBlaze B2 Bucket from Cloudflare Transform Rules

Access Private BackBlaze B2 Bucket from Cloudflare Transform Rules

Original post from my Blog.

Cloudflare now offers free Transform Rules, so BackBlaze B2 users can use URL Rewrite rules to hide bucket name and provide access to private bucket.

TLDR: Create a URL Rewrite Rule and hide your bucket name. If your bucket is private, you also need to create a Cloudflare Worker, copy the code and fill in the config.

Hide Bucket Name

URL Rewrite is quite simple. In Cloudflare dashboard, choose your domain -> Rules -> Transform Rules. Create a new Rewrite URL rule. It simply matches the domain, and dynamically rewrites to concat("/file/<YOUR_BUCKET_ID>", http.request.uri.path).

url-rewrite-rule

Access Private Bucket

If you want to access your private bucket from Cloudflare. You first need to create a BackBlaze Application Key. You need the keyID and applicationKey to generate a token, which is used to access files from the private bucket.

You can read the B2 documentation about how to generate the token. We can use the token to download files from private buckets in two ways: add Authorization in request header or add Authorization param in request query. The first header option needs additional "Request Header Modification rule" and the response cannot be cached by Cloudflare, so we use the second method.

Edit the URL Rewrite rule we created to hide bucket name, in Query choose "Rewrite to", select Static and enter Authorization=<Token>.

The problem is that the token is only at most 24 hours. So I write this script to automatically create token and update the rule. You can create a Cloudflare worker to do the job and set Triggers to run every 30 minutes.

Create a Cloudflare worker. The name does not matter since we use the Trigger of the worker. Open the quick editor. Copy and paste the code.

The configs you need to fill in:

  • B2KeyID and B2AppKey by create a BackBlaze Application Key.

  • B2BucketID: Your BackBlaze B2 bucket ID.

  • cfDomain: The domain you want to access b2 from.

  • cfAuthEmail: Your Cloudflare email.

  • cfAuthKey: Your Global API Key from API Tokens.

  • cfZoneID: Your domain zone ID from the overview dashboard of your domain.

  • cfRulesetID: You need first create the URL Rewrite rule. Then use the following curl command to view all the rulesets you have. Find the ruleset with phase http_request_transform. Its id is the cfRulesetID we want.

    curl -X GET \
      -H "X-Auth-Email: <cfAuthEmail>" \
      -H "X-Auth-Key: <cfAuthKey>" \
      "https://api.cloudflare.com/client/v4/zones/<cfZoneID>/rulesets"
  • cfRuleID: Once you have the cfRulesetID, use the following curl command to view the rules you have in the rule set. Find the rule you created in Cloudflare dashboard, its id is the cfRuleID we want.

    curl -X GET \
      -H "X-Auth-Email: <cfAuthEmail>" \
      -H "X-Auth-Key: <cfAuthKey>" \
      "https://api.cloudflare.com/client/v4/zones/<cfZoneID>/rulesets/<cfRulesetID>"

Once you fill in the constants, you can by Save and Deploy and then Run.

const config = {
// B2 config
B2KeyID: '',
B2AppKey: '',
B2BucketID: '',
// Cloudflare config
cfDomain: '',
cfAuthEmail: '',
cfAuthKey: '',
cfZoneID: '',
cfRulesetID: '',
cfRuleID: ''
}
addEventListener("fetch", event => {
event.respondWith(handleRequest(event.request))
})
addEventListener("scheduled", event => {
event.waitUntil(updateRule(config))
})
const getB2Token = async (config) => {
const { B2KeyID, B2AppKey } = config
const res = await fetch('https://api.backblazeb2.com/b2api/v2/b2_authorize_account', {
headers: {
'Authorization': 'Basic ' + btoa(B2KeyID + ":" + B2AppKey)
}
})
const data = await res.json()
return data.authorizationToken
}
const updateRule = async (config) => {
const b2Token = await getB2Token(config)
const { B2BucketID, cfDomain, cfAuthEmail, cfAuthKey, cfZoneID, cfRulesetID, cfRuleID} = config
const res = await fetch(`https://api.cloudflare.com/client/v4/zones/${cfZoneID}/rulesets/${cfRulesetID}/rules/${cfRuleID}`, {
method: 'PATCH',
headers: {
'X-Auth-Email': cfAuthEmail,
'X-Auth-Key': cfAuthKey
},
body: `{
"description": "B2 ${B2BucketID}",
"action": "rewrite",
"expression": "(http.host eq \\\"${cfDomain}\\\")",
"action_parameters": {
"uri": {
"path": {
"expression": "concat(\\\"/file/${B2BucketID}\\\", http.request.uri.path)"
},
"query": {
"value": "Authorization=${b2Token}"
}
}
}
}`
})
const data = await res.text()
console.log(data)
return data
}
async function handleRequest(request) {
const data = await updateRule(config)
return new Response(data)
}
@lzjluzijie
Copy link
Author

It seems that the cf-cache-status for private buckets is always BYPASS instead of normal HIT/MISS. Not sure why this happens.

@Gocnak
Copy link

Gocnak commented Dec 15, 2021

@lzjluzijie It seems like you may need to set a bucket setting to have this cache properly, see Step 5 of this Blog

(Putting in image form for backup as well):
image

@lzjluzijie
Copy link
Author

@Gocnak I tried setting cache-control but the cf-cache-status is still BYPASS, even my bucket is public now. I suppose this Cloudflare force BYPASS when request header is modified.

$ curl -i https://b2.halu.lu/Snipaste_2021-11-08_16-50-51.png
HTTP/2 200
date: Thu, 16 Dec 2021 01:57:34 GMT
content-type: image/png
content-length: 180176
x-bz-file-name: Snipaste_2021-11-08_16-50-51.png
x-bz-file-id: 4_z96de7f2c45e325df7dd90118_f1194ee1354ce4124_d20211210_m053044_c002_v0001133_t0002
x-bz-content-sha1: 1ea2f1d7f7c6ed4d56336d0f8e0e11960c5b64a9
x-bz-upload-timestamp: 1639114244000
cache-control: max-age=14400
x-bz-info-src_last_modified_millis: 1636361453638
cf-cache-status: BYPASS
accept-ranges: bytes
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=715he%2FTAA1PAg6FgOiF6zRnb8ODWt4ANoompS%2BxNR6%2BOtc3cr45QGLCnIcNfbVJMLd%2FOIyIOTExUU0Aum%2B97OO6L3F6iUaL3gFTRTkDmLaPYOGB5BDP1so5gDSFx"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 6be44d2dbec37e47-LAX
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400

@Gocnak
Copy link

Gocnak commented Dec 16, 2021

Yeah I think any Authorization or other headers added to the request makes CF think it's a private access request. Try removing the header write?

@lzjluzijie
Copy link
Author

@Gocnak You are right! From B2 document, we can actually put the Authorization token in the query params, so no Authorization header rewrite is needed. I will update the article.

@rocc-o
Copy link

rocc-o commented Dec 29, 2021

I wonder, since we use the trigger of the worker to run every 30 minutes, this will generate a load of requests, in addition to those to serve the images, which may not suitable for a workers free plan with max 100,000 / day requests. Am I correct? Better to go with workers paid plan in this case?

@lzjluzijie
Copy link
Author

@rocc-o Well, every 30 minutes is only 48 requests per day.

@rocc-o
Copy link

rocc-o commented Jan 9, 2022

Yes, in fact it is feasible. I will try your solution in the next days. Thanks!

@feliperaul
Copy link

@lzjluzijie This worked great, thank you.

However, if you don't mind me asking, after setting all this up, I realized I was just effectively transforming a private bucket into a public one, since the authorization query param is being added automatically by cloudflare.

What's exactly the use case here? Shouldn't we just create a public bucket to achieve the same desired effect?

@lzjluzijie
Copy link
Author

@feliperaul If you use a public bucket, attackers can generate huge traffic to download your files from backblaze directly, which will cost a lot.

@feliperaul
Copy link

@lzjluzijie Just to be extra clear, my desired use-case would be something like this: have a vanity url configured in Cloudflare DNS (like static.mydomain.com) that would allow me to access private buckets files, but requiring signed URLs.

Rails ActiveStorage has S3 integration so it automatically uses the B2 api and generates URLs like

https://my-bucket-name.s3.us-west-004.backblazeb2.com/filename?response-content-disposition=inline%3B%20filename%3D%myfile.pdf%22%3B%20filename%2A%3DUTF-8%27%27my-file.pdf&response-content-type=application%2Fpdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=credentialxyzdoijaojo%2Fus-west-004%2Fs3%2Faws4_request&X-Amz-Date=20220119T183559Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=somesignaturexyz

I would like to be able to access that file at https://static.mydomain.com/filename?response-content-disposition=inline%3B%20filename%3D%myfile.pdf%22%3B%20filename%2A%3DUTF-8%27%27my-file.pdf&response-content-type=application%2Fpdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=..., so not only I have a vanity URL but it also uses Cloudflare to fetch the file, so there's no bandwith charge in that download (since B2 and Cloudflare have the Bandwith Alliance deal).

@lzjluzijie
Copy link
Author

lzjluzijie commented Jan 19, 2022

@feliperaul I think my solution does not work with signed URL, it simply exposes a private bucket to behave like a public bucket using Cloudflare.

@feliperaul
Copy link

@lzjluzijie Thank you, I tried using CNAME in Cloudflare but for some reason it returns unauthenticated even with the pre-signed URLs (it works fine, tough, if I access using the pre-signed URL with the B2 hostname).

I've opened a support ticket about this.

@Lejo1
Copy link

Lejo1 commented Feb 1, 2022

What about using a key from the b2_get_download_authorization-Api call?
It can last up to one week and allows custom override of Headers like CacheControl

@ka3hun9
Copy link

ka3hun9 commented Mar 23, 2023

What about using a key from the b2_get_download_authorization-Api call? It can last up to one week and allows custom override of Headers like CacheControl

https://help.backblaze.com/hc/en-us/articles/360010017893-Delivering-Private-Backblaze-B2-Content-Through-Cloudflare-CDN

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment