Documentation:
- https://docs.aws.amazon.com/general/latest/gr/aws_sdk_cryptography.html
- https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
Ruby:
- https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/s3-example-client-encryption.html
- https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3.html
- https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Encryption.html
Go:
- https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/s3-examples-encryption.html
- https://docs.aws.amazon.com/sdk-for-go/api/service/s3/
- https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3crypto/
Objects are encrypted in the S3 client and then uploaded to S3. The objects can additionally be encrypted on the server side as well, if desired. If not encrypted on the server side, the S3 console will say "Encryption: None". The S3 console doesn't care whether or not the object used client-side encryption, you can still download the ciphertext.
The aws-cli and S3 Management Console does not support client-side encryption. They will just display the ciphertext.
Client-side encryption uses envelope encryption, meaning that the client generates an data key that is used to encrypt the object contents. The data key is then encrypted using the encryption key configured (RSA key or KMS key). Not all SDKs support all ciphers (see below). The encrypted data key is stored alongside the object as metadata. The result of this is that all objects are encrypted with a unique data key.
AWS also has an "Encryption SDK", which is separate and not interoperable with S3 client-side encryption. It is also not available for all languages (Ruby and Go are NOT supported at the moment). See https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/introduction.html
Google Cloud has an implementation of the S3 API. The "Interoperability API" has to be enabled.
When using RSA, there is no dependency on AWS KMS. I doubt that Google Cloud KMS has any compability with AWS KMS. It is perfectly possible to store objects in Google Cloud and use AWS KMS to encrypt the data key. The example code here supports uploading the objects to both AWS S3 and Google Cloud Storage, when using AWS KMS to encrypt the data key.
Azure does not seem to support the S3 API. I found several examples where people use proxies and are able to use the AWS SDK that way. I did not try this.
Create a Storage Bucket in Google Cloud. Remember to enable the "Interoperability API". Generate an access key.
Set up a s3test_gcloud profile in ~/.aws/credentials. To use KMS, also set up an s3test_aws profile with a user that has access to the KMS key (and optionally to an S3 bucket).
[s3test_gcloud]
region=does-not-matter
aws_access_key_id=ABC
aws_secret_access_key=XYZ
[s3test_aws]
region=us-west-2
aws_access_key_id=ABC
aws_secret_access_key=XYZ
https://docs.aws.amazon.com/general/latest/gr/aws_sdk_cryptography.html
The Ruby SDK can use:
- AES/ECB key wrap (not recommended)
- RSA key wrap
- AES/CBC content encryption (Encryption Only mode)
- AES/GCM content encryption (Strict Authentication mode)
- AES/CTR content encryption (Authenticated mode only used for decrypting in range GETs)
The Go SDK can use:
- AES/CBC content encryption (Encryption Only mode)
- AES/GCM content encryption (Strict Authentication mode)
The Ruby SDK currently is not configurable to encrypt with AES/GCM, but it can decrypt it. When using KMS as a cipher provider, it will always use AES/CBC to encrypt.
Newer ciphers, such as AES/CBC and AES/GCM, use KMS to encrypt the data key.
There seem to be some support to extend the Go SDK to support additional ciphers and additional KMS implementations, but I did not experiment with this.
openssl genrsa -out private_key.pem 2048
openssl rsa -in private_key.pem -outform PEM -pubout -out public_key.pem
This investigation was just a short spike to understand how the AWS SDKs implement client-side encryption.
If there is a desire to store encrypted data in places other than S3, then the AWS SDK will be difficult to use.
In my opinion, this can be considered on a case-by-case basis. For data that will be stored in other places, I think Vault should be considered instead. Vault has SDKs for a lot of languages, and I suspect most of them have good feature parity.