Storing Google Cloud Platform (GCP) Credentials in AWS Secrets Manager for Apache Airflow to connect to BigQuery
This took me a while to figure out by looking at the code & docs. Most Google searches were directing me to useless articles (usually about storing the creds in GCP Secrets Manager)
- Create a service account (or other account)
- Create a new key and download the JSON file
- Open the file & copy the entire JSON into the clipboard
- Create an AWS Secret (name it appropriately, let's say
airflow/connection/mydag/gcp
- Set the key to
extra
- Set the value to the contents to:
- Set the key to