There is a need for custom format in the backend for gateway sse encryption for multipart operations, primarily because of limitations of s3 multipart API protocol and the need for size of each encrypted part to be maintained for successful decryption.
multipart gateway sse encryption creates an object per object-part at the backend like below:
- Gateway receives a
new-multi-part
request forbucket/foo
by client. -> Gateway creates the object-prefixbucket/foo
and stores encryption metadata forfoo
asbucket/foo/dare.meta
This file will itself be encrypted to avoid leaking information. - Gateway receives a new object part from the client. E.g. part 1.
-> Gateway creates the object
bucket/foo/part.1
after encrypting part with part encryption key derived frombucket/foo/dare.meta
For any part received by the client the gateway simply creates an object identified by e.g. the part.ID under the object prefix. - Gateway receives a complete multipart request
-> Gateway updates the
bucket/foo/dare.meta
with each part's info and part size. GET operations
- gateway composes the object content using dare.meta and parts bucket/foo/part.1...
- object can be manipulated only using the s3 gateway. A client that accesses backend directly will see the encrypted parts listed separately.
- part info needs to be filtered from listing.
- dare.meta
{
"version": "1.0.0",
"format": "gw",
"stat": {
"size": 68190720,
"modTime": "2018-08-29T05:52:55.315554552Z"
},
"meta": {
"X-Minio-Internal-Encrypted-Multipart": "",
"X-Minio-Internal-Server-Side-Encryption-Iv": "1HiTOW8iVd8LaxHBoGYasrNmuXXdH5ggEhczWuDPzZM=",
"X-Minio-Internal-Server-Side-Encryption-Seal-Algorithm": "DAREv2-HMAC-SHA256",
"X-Minio-Internal-Server-Side-Encryption-Sealed-Key": "IAAfAK240adWxEsX69KcoYLRBlsvBOV9GsDWmB4R5jk2f1yZC6pwm/Tsg6PpEj92gFdwBrx0Dey07EQhy7Jv9A==",
"content-type": "text/plain; charset=utf-8",
"etag": "36fd294825fd6e126e62e999cfe0222c-2"
},
"parts": [
{
"number": 1,
"name": "part.1",
"etag": "e3779dd073f916166977c637de6c438b",
"size": 67141632
},
{
"number": 2,
"name": "part.2",
"etag": "04596ad16133658ab4d8264a66cf6d8f",
"size": 1049088
}
]
}
I don't see how we can achieve this... For SSE-C it is possible but for SSE-S3 the key is embedded into the
dare.meta
. (SeeX-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key
) Actually it's not needed to encrypt the metadata since it also public in server mode.So
meta.dare
MUST not contain confidential information - like the object metadata sent by the client. (S3 specification)However (at the moment)
dare.meta
does not leak any information AFAICS which is not already public. Nevertheless using "double encryption" thedare.meta
can of course be encrypted using the SSE capabilities of the backend...