Skip to content

Instantly share code, notes, and snippets.

@Dominik389
Dominik389 / dataproc_cluster.tf
Last active August 25, 2022 09:59
The script creates a Dataproc cluster with two worker nodes and a master node in the GCP. This cluster is scaled by a scaling policy and contains the optional components Zookeeper and HBase. Furthermore, Kafka is installed through an initialization action. The exact procedure is explained within the file.
# To use this file, you need to install Terraform. How to do that can be found here: https://learn.hashicorp.com/tutorials/terraform/install-cli.
# To use this file, put it in a directory and name the file main.tf. This will tell Terraform to use this file.
# Then create a subdirectory called key.
# Get a service account key from your GCP project. To do this, go into your GCP project. Then go to IAM -> Service Accounts -> Compute Engine Standard Service Account -> Key -> Add Key -> Create New Key -> JSON.
# The key will now be downloaded to your computer.
# Load a service account key from your GCP project into the key directory and rename the service account key to GCPkey.json.
# Then replace all values that are in this format: <<< value >>>.
# Now start a shell and go to the directory where this file is located as main.tf.
# Run the command "terraform init" there. Terraform will now load everything to use its providers.
# After Terraform has been initialized, run the "terraform apply" command.