Secrets such as aws key and secret, google service account json, database id and password etc. can be easily encrypted and decrypted with https://cloud.google.com/kms/. Cloud KMS does not directly store secrets. It can encrypt secrets that you store elsewhere, i.e. the key itself is stored within KMS.
Let's illustrate with a real world example step by step. We can encrypt and decrypt a service account json file for compute instances. These instances are a part of an elasticsearch cluster. The google cloud admin creates the service account. The service account is used by the Terraform to provision the compute instances as shown in https://www.terraform.io/docs/providers/google/r/compute_instance.html. The developers want a copy of the service account 's json file so that they can develop & test with the elasticsearch cluster. The google cloud admin creates the plain text service json file, where and how to store it safely and securely? Storing on admin's laptop is not 100% safe and secure. Here are the steps that leverage Cloud KMS. For simplicity, we won't talk about key rotation here.
gcloud kms keyrings create dev_keyring --location global
gcloud kms keys create sa --location global --keyring dev_keyring --purpose encryption creates a key sa for encrypting google service account json file.
gcloud kms encrypt --location=global --keyring=dev_keyring --key=sa --plaintext-file=elasticsearch_svc_account.json --ciphertext-file=elasticsearch_svc_account.json.enc
At this point, we can delete the plain text file elasticsearch_svc_account.json from the laptop.
export GOOGLE_PROJECT=$(gcloud config get-value project)
export ENV=dev
gsutil cp elasticsearch_svc_account.json.enc gs://${GOOGLE_PROJECT}-secrets-${ENV}/
Where to store the encrypted secrtes? They can be stored in a GCS bucket or any configuration managed system's data storage such as a chef data bag, a salt pillar or an ansible vault,or HashiCorp 's Vault https://cloud.google.com/solutions/using-vault-for-secret-management. In our Terraform example, it is stored in a GCS bucket.
export GOOGLE_PROJECT=$(gcloud config get-value project)
export ENV=dev
gcloud kms decrypt --location=global --keyring=dev_keyring --key=sa --plaintext-file=/dev/stdout --ciphertext-file=<(gsutil cat gs://${GOOGLE_PROJECT}-secrets-${ENV}/elasticsearch_svc_account.json.enc)
In our Terraform example, we can use the Terraform external data provider as https://github.com/GoogleCloudPlatform/terraform-google-vault/blob/master/main.tf#L95 to download and decrypt the elasticsearch_svc_account.json.enc onto the console. The cloud admin can give the service json to the developer who needs it via a secure channel.
Thank You!