An instructional document by Robin H Johnson robin.johnson@dreamhost.com. I wrote much of the staticsites functionality of Ceph-RGW, during during late 2015 and early 2016, based on an early prototype by Yehuda Sadeh (yehudasa). It was written for usage at Dreamhost, but developed in the open for community improvement.
It is fully functional as of Jewel v10.2.3 plus PR11280 (ceph/ceph#11280). Prior to that, neither the non-CNAME nor CNAME-to-service modes will function correctly.
These configuration files represent how to quickly set up RGW+HAProxy for staticsite serving. I've tried to make them more readable, without leaving out too many details. You are strongly recommended to run a seperate RGW instance for staticsites, on a DIFFERENT outward-faciing IP than your normal instance (and in fact, certain functionality is not supported without it).
In place of using HAProxy, you could run the second rgw instance on port 80, just with a different IP than the primary instance.
objects-region.domain.com. IN A 192.0.2.10
objects-region.domain.com. IN AAAA 2001:DB8::192:0:2:10
*.objects-region.domain.com. IN CNAME objects-region.domain.com.
objects-website-region.domain.com. IN A 192.0.2.20
objects-website-region.domain.com. IN AAAA 2001:DB8::192:0:2:20
*.objects-website-region.domain.com. IN CNAME objects-website-region.domain.com.
- Access on 'http://bucket1.objects-website-region.domain.com', bucket=
bucket1 - DNS entry:
*.objects-website-region.domain.com. IN CNAME objects-website-region.domain.com. - HTTPS will work, as the SSL wildcard matches exactly one level.
- Access on 'http://www.example.com.objects-website-region.domain.com', bucket=
www.example.com - DNS entry:
*.objects-website-region.domain.com. IN CNAME objects-website-region.domain.com. - HTTPS will NOT work, as the SSL wildcard matches exactly one level.
- Access on 'http://www.example.com/', bucket=
bucket2 - DNS entry:
www.example.com. IN CNAME bucket2.objects-website-region.domain.com. - HTTPS will work if the proxy has a certificate for
www.example.com. - Special note(Ceph): this is functionality UNIQUE to Ceph RGW. It is NOT supported by AWS S3. AWS requires the bucket name must match the hostname, and will not otherwise work. It is enabled with
rgw_resolve_cname, and requires that the S3 server be able to resolve the CNAME from it's view of DNS (this can cause problems with split-horizon DNS).
- Access on 'http://www.example.com/', bucket=
www.example.com - DNS entry:
www.example.com. IN CNAME www.example.com.objects-website-region.domain.com. - HTTPS will work if the proxy has a certificate for
www.example.com.
- Access on 'http://example.com/', bucket=
example.com - DNS entry:
example.com. IN A 192.0.2.20 - DNS entry:
example.com. IN AAAA 2001:DB8::192:0:2:20 - HTTPS will work if the proxy has a certificate for
example.com. - Special note: This variant is required for any DNS name that has other non-CNAME records, like SOA/NS/MX/TXT etc.
- Access on 'http://www.example.com/', bucket=
www.example.com - DNS entry:
www.example.com. IN CNAME objects-website-region.domain.com. - HTTPS will work if the proxy has a certificate for
www.example.com. - Special note(AWS): This DNS is supported as 'legacy' by AWS S3 only, and is not recommended.
Beyond just using rgw_dns_name and rgw_dns_s3website_name, you can use RGW zonegroup configuration to do the same thing, now with support for as many hostnames as you would like.
{
"api_name" : "regionname",
"default_placement" : "default-placement",
"endpoints" : [
],
"hostnames" : [
"objects-region.domain.com",
"objects-region.branding.com"
],
"hostnames_s3website" : [
"objects-website-region.domain.com",
"objects-website-region.branding.com"
],
"id" : "REGIONNAME",
"is_master" : "true",
"master_zone" : "REGIONNAME",
"name" : "REGIONNAME",
"placement_targets" : [
{
"name" : "default-placement",
"tags" : []
}
],
"realm_id" : "",
"zones" : [
{
"bucket_index_max_shards" : 31,
"endpoints" : [],
"id" : "CENSORED",
"log_data" : "false",
"log_meta" : "true",
"name" : "CENSORED",
"read_only" : "false"
}
]
}
Your rgw_dns_name/rgw_dns_s3website_name entries and the entries from the zonegroup must NOT contain any overlaps.
No complete name should be a trailing of any other name, assuming implicit leading periods.
If these names are configured:
['s3.abc.com', 's3-website.abc.com', 'website-s3.abc.com']
They are treated as:
['.s3.abc.com', '.s3-website.abc.com', '.website-s3.abc.com']
Adding any of the following would cause an overlap:
'abc.com'- all entries overlap this'alt.s3.abc.com'- overlaps'.s3.abc.com'
If you MUST limit yourself to a single RGW instance, here's what you're going to lose:
- http://example.com/ Alternative hostname to matching long bucket, without CNAME. The single RGW instance will not be able to determine what API (
s3ors3website) to use, because it's perfectly valid to use a browser with the normal S3 API to GET/HEAD objects.
If you MUST limit yourself to single public IP, and you can put the two RGW instances behind it, here's what you're going to lose:
- If you configure your HAProxy to direct anything that DOES NOT match
rgw_dns_name/zonegrouphostnamesto the staticsites instance.
- Any normal S3 access with
CallingFormat=VHostwill not work, as it will end up on the s3website API.
- If you configure your HAProxy to direct anything that DOES match
rgw_dns_website_name/zonegrouphostnames_s3websiteto the staticsites instance.
- You will only be able to use hostnames that match those DNS wildcards.
My radosgw is not responding "7480 after 0 ms: Couldn't connect to server"
sample error
2024-03-28T12:38:53.915+0300 7f61215c46c0 0 --2- 172.12.1.34:0/1509191733 >> [v2:172.12.1.34:3300/0,v1:172.12.1.34:6789/0] conn(0x56551127ace0 0x565511312220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -13