Google Cloud Notes

google_cloud_notes.txt

Set up for the first time:
> gcloud config set container/use_client_certificate True
> gcloud container clusters get-credentials orchestrator --zone europe-west1-d --project prodigy-finance-edw

First log in to google:
> gcloud auth login

Check the airflow ui:
> kubectl port-forward af-ui 8080:8080
Then go to: http://localhost:8080

Build and deploy a new airflow docker file
* First mke your changes in the github repo: https://github.com/DotModus/prodigy-edw
> docker build -t eu.gcr.io/prodigy-finance-edw/airflow-main:<X.X> prodigy-docker-files
> gcloud docker -- push eu.gcr.io/prodigy-finance-edw/airflow-main:<X.X>

Check logs:
> kubectl proxy
Then go to: http://localhost:8001/ui

Connect to datalab:
In the console
> datalab connect datalab-test-laura
The click on Web Preview > preview on port 8081

To get into one of the pods
> kubectl exec -ti af-ui -c ui bash
> kubectl exec -ti af-worker-0 bash