gateway sse backend format.md

Gateway SSE backend format

There is a need for custom format in the backend for gateway sse encryption for multipart operations, primarily because of limitations of s3 multipart API protocol and the need for size of each encrypted part to be maintained for successful decryption.

multipart operation with sse at gateway

multipart gateway sse encryption creates an object per object-part at the backend like below:

1. Gateway receives a new-multi-part request for bucket/foo by client. -> Gateway creates the object-prefix bucket/foo and stores encryption metadata for foo as bucket/foo/dare.meta This file will itself be encrypted to avoid leaking information.
2. Gateway receives a new object part from the client. E.g. part 1. -> Gateway creates the object bucket/foo/part.1 after encrypting part with part encryption key derived from bucket/foo/dare.meta For any part received by the client the gateway simply creates an object identified by e.g. the part.ID under the object prefix.
3. Gateway receives a complete multipart request -> Gateway updates the bucket/foo/dare.meta with each part's info and part size. GET operations

• gateway composes the object content using dare.meta and parts bucket/foo/part.1...

Limitations:

• object can be manipulated only using the s3 gateway. A client that accesses backend directly will see the encrypted parts listed separately.
• part info needs to be filtered from listing.

Custom dare.meta format

• dare.meta

{
  "version": "1.0.0",
  "format": "gw",
  "stat": {
    "size": 68190720,
    "modTime": "2018-08-29T05:52:55.315554552Z"
  },
  "meta": {
    "X-Minio-Internal-Encrypted-Multipart": "",
    "X-Minio-Internal-Server-Side-Encryption-Iv": "1HiTOW8iVd8LaxHBoGYasrNmuXXdH5ggEhczWuDPzZM=",
    "X-Minio-Internal-Server-Side-Encryption-Seal-Algorithm": "DAREv2-HMAC-SHA256",
    "X-Minio-Internal-Server-Side-Encryption-Sealed-Key": "IAAfAK240adWxEsX69KcoYLRBlsvBOV9GsDWmB4R5jk2f1yZC6pwm/Tsg6PpEj92gFdwBrx0Dey07EQhy7Jv9A==",
    "content-type": "text/plain; charset=utf-8",
    "etag": "36fd294825fd6e126e62e999cfe0222c-2"
  },
  "parts": [
    {
      "number": 1,
      "name": "part.1",
      "etag": "e3779dd073f916166977c637de6c438b",
      "size": 67141632
    },
    {
      "number": 2,
      "name": "part.2",
      "etag": "04596ad16133658ab4d8264a66cf6d8f",
      "size": 1049088
    }
  ]
}

This file will itself be encrypted to avoid leaking information.

I don't see how we can achieve this... For SSE-C it is possible but for SSE-S3 the key is embedded into the dare.meta. (See X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key) Actually it's not needed to encrypt the metadata since it also public in server mode.
So meta.dare MUST not contain confidential information - like the object metadata sent by the client. (S3 specification)

However (at the moment) dare.meta does not leak any information AFAICS which is not already public. Nevertheless using "double encryption" the dare.meta can of course be encrypted using the SSE capabilities of the backend...