Skip to content

Instantly share code, notes, and snippets.

@vuongpd95

vuongpd95/gcp-command-cheatsheet.md

Last active October 31, 2020 06:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vuongpd95/40cd36204140db36f42ae1518c12b83c to your computer and use it in GitHub Desktop.
Save vuongpd95/40cd36204140db36f42ae1518c12b83c to your computer and use it in GitHub Desktop.
Download ZIP
GCP command cheatsheet
Raw
gcp-command-cheatsheet.md

GCP Data flow

For VM instance, it's essential that:

  • Before a VM instance can be created, compute API should be enabled gcloud services enable compute.googleapis.com
  • zone and machine-type are required in order to create a VM instance
  • Metadata can be passed to the instance by using --metadata={{ attribute name }}={{ attribute value }} which then can be retrieved by curl -H "Metadata-Flavor: Google" http://http://metadata.google.internal/computeMetadata/v1/instance/attributes/{{ attribute name }} while inside the instance.
  • To set startup script of a VM instance from a file --metadata-from-file startup-script={{ Relative path to the startup script }}. But, do remember, startup-script is just an attribute so we can also do --metadata startup-script='...'. startup-script can be read by curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/attributes/startup-script
  • For the VM instance to be able to use any GCP services, it needs to have the correct scopes (https://cloud.google.com/sdk/gcloud/reference/alpha/compute/instances/set-scopes#--scopes). Set the scopes using --scopes=default,{{ extra scopes }}

About Google VPC Network

  • VPC Network API is already enabled when we enable GCE (Google Compute Engine)
  • gcloud commands for VPC Network is prefix by compute such as gcloud compute networks list
  • To create a new VPC: gcloud compute networks create exploration-network --bgp-routing-mode=global --description='just a test' --mtu=1500 --subnet-mode=auto
  • To delete a VPC: gcloud compute networks delete {{ name }} {{ name }} BUT all of their firewall-rules needs to be deleted first using gcloud compute firewall-rules delete {{ name }} {{ name }}; Get firewall-rules names from gcloud compute firewall-rules list
  • A VPC won't work without a firewall: gcloud compute firewall-rules create exploration-network-firewall --network exploration-network --allow tcp:22,tcp:3389,icmp

More examples:

# Create regional custom network
gcloud compute networks create sel-network --subnet-mode=custom --bgp-routing-mode=regional
# Create a subnet for the network
gcloud compute networks subnets create us-central1-subnet --range=172.168.0.0/24 --network=sel-network --region=us-central1
# Create firewall rule for the network
gcloud compute firewall-rules create sel-network-fw --network=sel-network --allow tcp:22,tcp:3389,icmp

GCP IAM

  • A permission is the lowest form of authorization, usually correspond to an API call to a service.

  • A role is a collection of permissions.

  • Service account is an identity that used by machine to access to Google Cloud Services. It can have roles. It acts like a resource means if a real user want to use a service account, he/she needs permission.

  • A real user (identified by your google account) needs to be granted roles or permissions to acccess GCP.

  • Using group is an effective way to manage your organization since group is a collection of users and service accounts. A group is identified by an email address, has an IAM policy. By adding an user to a group, you effectively granted all permissions that associated with that group. When the user is retired, by removing the user from his group, you remove all his permissions to the project.

  • There is a resource hierarchy, Organization -> Folder -> Project -> Resources. Policy (a JSON file that specified what role an user has on a scope of resources) is inherited from top to bottom

GCP SDK

# List all projects
gcloud projects list

# List current cloud shell confg
gcloud config list

# Create a new project
gcloud projects create services-exploration-labs-1211 --name="Services Exploration Labs"

# Attach project to a billing account. CAREFUL, this needs some time to propagate to the services
gcloud beta billing projects link services-exploration-labs-1211 --billing-acount={{ billing account id }}

# CAREFUL! Remember to set the current project since cloud shell is used for every projects that you have
gcloud config set project {{ project-id }}

# Get the current project
gcloud config get-value project

# Get all configurations
gcloud config configurations list

# Create a new configurations and switch to it
gcloud config configurations create {{ config name }}

# Delete a configuration
gcloud config configurations delete {{ config name }}

# Get configuration description
gcloud config configurations describle {{ config name }}

# List all services
gcloud services list 

# Enable a service
gcloud services enable compute.googleapis.com

# List all GC VMs
gcloud compute instances list

# Create an VM instance
gcloud compute instances create --machine-type={{ machine type }} test-vm

# Create an VM instance with scopes, https://cloud.google.com/sdk/gcloud/reference/alpha/compute/instances/set-scopes#--scopes
# Instance metadata will be written to http://metadata.google.internal/computeMetadata/v1/instance/attributes/{{ attr name }}
gcloud compute instances create --machine-type={{ machine type }} --scopes=default,storage-rw test-vm \
  --metadata-from-file startup-script={{ relative path }} \
  --metadata={{ attr name }}={{ attr value }}
  
# Inside the instance, retrieve metadata by
curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/attributes/{{ attr name }}

# Delete
gcloud compute instances delete test-vm

# List all compute machine types, zones, regions
gcloud compute {{ machine-types|zones|regions }} list
# e.g.
gcloud compute machine-types list --filter="NAME=f1-micro AND ZONE~asia-southeast"

# Set defauls for VM instance, CAREFUL! We can set an invalid zone that not in the list
gcloud config set compute/zone {{ zone }}
gcloud config set compute/region {{ region }}

# SSH to a compute VM
gcloud compute ssh {{ vm name }}

# Query for meta of the instance. CAREFUL! You need to be on the instance
curl -H "Metadata-Flavor:Google" http://metadata.google.internal/computeMetadata/v1/project/project-id

GCS bucket

# In Cloud Shell
# Add -m to gsutil to perform the command multi-thread

# A bucket can be created using cloud shell (https://cloud.google.com/storage/docs/locations)
gsutil mb -l {{ locations }} gs://{{ bucket-id }}

# Get labels of a bucket
gsutil label get gs://{{ bucket-id }}

# Set a json file as label for a bucket
gsutil label set label-file.json gs://{{ bucket-id }}

# Change a label 
gsutil label ch -l "label-key:label-val" gs://{{ bucket-id }}

# Delete a label
gsutil label ch -d "label-key" gs://{{ bucket-id }}

# Change labels like this works too! Label changes are executed from left to right
gsutil label ch -d "label-key" -l "label-key:label-val" gs://{{ bucket-id }}

# Check if bucket versioning is enabled
gsutil versioning get gs://{{ bucket-id }}

# Set bucket versioning status, turn off versioning doesn't delete versioned objects
gsutil versioning set {{ on|off }} gs://{{ bucket-id }}

# Use ls -a to see version of bucket objects. 'a' stands for archive
gsutil ls -a gs://{{ bucket-id }}/file-name.txt

# Restore a deleted object simply by copying the versioned name of that object to its original name. This create a brand-new object with different version number. Deleting the versioned old object won't delete the new object
gsutil cp gs://{{ bucket-id }}/file-name.txt#{{ v1 }} gs://{{ bucket-id }}/file-name.txt 

# Get current live version of an object.
# I don't know, tell me in the comment section if you know! Thanks!

# Get access control list (ACL) of an object
gsutil acl get gs://{{ bucket-id }}/file-name.txt

# Set ACL of an object (more using gsutil acl --help)
gsutil acl get gs://{{ bucket-id }}/file-name.txt > acl-file-name.json
# Make changes to acl-file-name.json
gsutil acl set acl-file-name.json gs://{{ bucket-id }}/file-name.txt

# Make an object publicly readable
gsutil acl ch -u AllUsers:R gs://{{ bucket-id }}/file-name.txt
# To Revert
gsutil acl set private gs://{{ bucket-id }}/file-name.txt

# List all buckets
gsutil list

# List all objects of a bucket
gsutil list gs://{{ bucket-id }}

# Move object inside a bucket
gsutil mv gs://{{ bucket-id }}/file-path.txt gs://{{ bucket-id }}/file-path-alt.txt

# Move object between bucket works too!
gsutil mv gs://{{ bucket-id }}/file-path.txt gs://{{ bucket-id-1 }}/file-path.txt

# Copy all objects from bucket-A to bucket-B in a "flat" way (not moving directory structure)
# If bucket-A has 2 objects with the same name belong to different directories, the outer most object wins
gsutil cp gs://bucket-A/** gs://bucket-B

# Use cp -r to also copy the directory structure
gsutil cp -r gs://bucket-A/* gs://bucket-B

# CAREFUL! This will create a directory named bucket-B in bucket-a
gsutil cp -r gs://bucket-A gs://bucket-B

# We can't create a directory but we can move an existing object into one that doesn't necessary exist
gsutil mv gs://{{ bucket-id }}/file-path.txt gs://{{ bucket-id }}/test-folder/file-path.txt 

# Delete all objects inside a bucket
gsutil rm -r gs://{{ bucket-id }}/*

# CAREFUL! Delete a whole bucket
gsutil rm -r gs;//{{ bucket-id }}

Tips & Summary:

  1. Give your bucket a short name
  2. GCS buckets work much like a file system with bucket list act as top level directories
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment