GCP Notes Core Infrastructure

                              Intro to GCP
- 4 main services:
  -  compute
  - storage
  - big data
  - ML

- Cloud Computing
  - On-demand self service
  - Broad Network access
  - Resource pooling
  - Rapid Elasticity
  - Measured Service

- Colocation
  - shared facilities
- Virtualization
  - virtual devices sepeartely managable from the real hardware.
- Severless
  - google switched to container based architecture.

- Every Company is a data company

- GCP computing architectures
  - Compute Engine: IaaS
  - Kubernetes Engine: Hybrid
  - App Engine: PaaS
  - Cloud Function: Serverless Logic
  - Managed Services: Automatic Elastic Storage
  - SaaS: software as a service, google search, gsuite etc.

- The Google Network
  - Responds from the closest POP inorder to provide lowest latency.

- GCP regions and zones
  - zone is deployment area for google cloud platform resources.
  - Zone is not a single physical building.
  - Zones are grouped in a region.
  - fast round latency amoung regions in a same zone.
  - deploy application in multiple regions to avoid single point of failure.
  - Google cloud storage store data redundantly in differnt locations with atleast 160Km apart.

- Environmental responsibility
  - 100% carbon neutral from 2007.

- Google helps customer in avoiding feeling like vendor lock.
  - Google stack driver for cloud monitoring.
  - GCP service are compatible with the open source projects.
  - Kubernetes lets user mix and match serveral microservices running on differnet clouds.

- Why choose GCP
  - Compute : Compute Engien, Kubernetes Engine, app engine, cloud functions
  - Storage: Bigtable, cloud storage, cloud sql, cloud spanner, cloud datastore

- Design for security:
  - Custom Designed by Google.

- Google Front End
  - For services that are available to internet.
  - GFE provides protection form ddos.
  - Red team excercises. 

- Budgets and Billing
  - Define budget limit on a billing account.
  - Billing export let you store billing expense.
  - Reports is a visual tool to monitor usage.
  - Quotas:
    - To avoid Over Consumption of resources.
    - rate quotas and allocation quotas:
      - Rate Quotas are reset after a specific time.
      - Allocation quota: 5 networks per project.
    - These can be increased by Google cloud support.

- Google Cloud Identity and Access Management
  - Implement least priviledge.
  - Org Notes (define polices here)
    - Folders
    - Projects
    - Resources
  - All gcp services are associated with a project
    - Project ID: Globally unique, chosen by you, immutable
    - Porject name: Not unique, chosen by you, Mutable
    - Project Number: Globally unique, Assigned by GCP, Immutable
  - Project IDs are human reable strings.
  - Organize projects into folders.
  - Resource in a folder inherit IAM policies from the folder.
  - Org note is needed for Folders. (Top of the heirarchy)
  - If you have Gsuite domain, GCP automatically makes a gcp note.

- Identity and Access Management
  - who can take actions on different polices.
  - who: User who can do something(Google account, Google group etc)
  - can do what: IAM role (permissions grouped together in a role)
  - 3 types of role: Primitive, Predifined, Custom.
  - Premitive: apply them to gcp project, owner, viewer and editor role.
  - Custom roles can't be used on project and folder level.
  - What if you want to give permissions to a Compute Engine virtual machine, rather than to a person?
    - Service account for vm who want access to a cloud storage.

- Interact with GCP:
  - 4 ways:
    - Cloud Platform Console.
    - Cloud Shell and SDK
    - Cloud Conolse and Mobile app
    - REST-Based API
  - GCP Console:
    - Web portal
    - Gives access to cloud shell.
  - Google Cloud SDK
    - gcloud:  
    - gsutil: Google cloud storage
    - bq: Big query.
  - Restful APIs
    - Use JSON.
    - GCP lets you on and off APIs.
    - API Explorer:
      - see what api are available and what version.
    - Cloud Client Librarires. (Community owned, may not available for all languages)
    - Google API client Libraries. (Use this older version when GC libraries are not available).
  - Console cloud app on Andoird and iOS.

- Cloud Marketplace (Formerly Google Cloud Launcher)
  - Some free some dont.
  - GCP doesn't update software after deployed.
  - For example: we can deploy LAMP stack directly.

- VPC (Virtual Private Cloud) 
  - subnets in any gcp region worldwide.
  - subnets can span accross zones.

- Compute Engine
  - Lets user run VM.
  - Create VMs with GCP console or gcloud.
  - Custom VM possilbe.
  - GPUs are available as well.
  - Presistent standard disk or SSD(Data wont last on ssd after vm terminates).
  - Pick a boot image
  - Define startup scripts if you like.
  - Disk snapshots as backup or helpful when migrating VM to differnet region.
  - Preemptible VM: Terminate when its resources are needed elsewhere.
  - Auto Scaling

- VPC has routing table (no need to manage by default)
  - Firewall rules using metadata(Tags).

- VPC Peering
  - Peering between two vpc of different projects.
  - Shared VPC: to manage which part of project access vpc in another project.

- Cloud Load Balancing
  - They dont run in vm. no need to scale manage.
  - single anycast IP frontends all backend instances.
  - cross region load balacning
  - pre-warning is not required.

- Internal Load Balancer
  - Load balancer for internal services without the need for a public load balancing IP.

- Free Google DNS 8.8.8.8

- Cloud DNS
  - programmable
  - millions of dns zones and records.

- Cloud CDN
  - low latency
  - CDN interconnect partner program.

- Interconnect other network with VPC
  - VPN connection with cloud router
  - Cloud Router let your cloud vpc and other network to share data over the vpn using the border gateway protocol.

- Direct Peering
  - Putting a router in the same public data centre as a Google Point of presence.

- Highest uptimes user should use Dedicated interconnect (Covered By SLAs).

- Cloud Storage
  - Not File storage.
  - Not block storage where OS manage data as a chunks of data.
  - Keep data that will be accessible using the unique keys.
  - Unique keys are often URL.
  - Fully Managed scalable service.
  - Cloud storage is not a File system.
  - Comprises of bucket.
  - Buckets store storage objects.
  - Storage objects are immutable.
  - Always encrypts data on server side.
  - Data in transit is encrypted by https.
  - Bucket have globally unique name.
  - choose location and region for the bucket.
  - Cloud IAM to manage access to bucket.
  - Access Control List (ACL) for finer permissions.
  - Enable object versioning if you want.
  - Lifecycle Management policy.
  - Classes of Cloud Storage:
    - Multi Regional (99.5)
    - Regional (99.90) 
    - Nearline (99.00) (Month)
    - Coldline (99.00) (Year)
  - Transfer data to Cloud storage:
    - Online Transfer
    - Storage Transfer Service
    - Trasnfer Appliance

- Cloud Bigtable
  - NoSQL big data database service
  - highly scalable.
  - Persistent Hash Table
  - High Throughput
  - Same Open source api as HBase.
  - Apache HBase Installation vs Big Table
    - Scaling is better with Big Table.
    - Easy to upgrade and restart.
    - Access control with IAM.
    - Powers many google core services.
  - Access ways:
    - API (HBase client)
    - Streaming (Hadoop Map reduce or spark)
    - Batch Processing

- Cloud SQL
  - Offers MySQL and PostgresSQLBeta
  - Cloud SQL vs vm deployment:
    - Replicate services b/w multiple zones.
    - On demand and on demand backup.
    - Encrypted and firewall access.
    - Accessible by other GCP services and other outer services.

- Cloud Spanner 
  - for horizontal scaleability but it is costly than cloud SQL.

- Cloud DataStore
  - NoSQL, highly scalable
  - Store Structured data from App Engine apps.
  - Can be used as an integration point between App Engine and Compute Engine.
  - Auto sharding and replication
  - Also offers transactions unlike cloud bigtable.
  - Best for semi-structured data.
  - Free daily quota.

- Google Kubernetes Engine
  - can be considered somewhere between IaaS(Compute Engine) and PaaS(App Engine).
  - Containers:
    - Independent scalability of work load like in a PaaS environment.
    - Abstraction Layer of hardware and os like in a IaaS environment.
  - Kubernetes:
    - orchestrates many containers on many hosts.
    - Use docker to make containers.
    - Lets you deploy containers on a set of nodes called a cluster.
    - Cluster is set of master components that control the system as a whole and a set of nodes that run container.
    - A node is a computing instance.
    - node is vm in gcloud.
  - GKE
    - Kubernetes as a managed service.
    - Create cluster with gke using the gcp console or cloud sdk.
    - pod, smallest deployable unit in kubernetes.
    - each pod has a unique IP and set of ports.
    - containers inside pod interact using localhost network interface.
    - Deployment is a group of replicas of the same pod.
    - A service is the fundamental way of Kubernestes to represent Load Balacning.
    - Service groups a set of pods together.
    - autoscale command to scale based on serveral configs like cpu usage.
    - config files can also be used to configure kubernetes
    - manage config file with any VCS to keep track of changes to infrastructure.
    - Rolling updates:
      - kubernetes create pods of new version and destroyes the previous ones only when the new one are up and running fine.

- Hybrid Cloud
  - Some services on-premises and some on cloud.
  - Anthos:
    - Hybrid and multi cloud management system.
    - Anthos framework rest on Kubernetes and GKE On-prem.
    - Anthos provides a rich set of tools for monitoring services on on-premises and in the cloud.
    - In software architecture, a service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between microservices, often using a sidecar proxy.
    - Cloud Interconnect extends your on-premises network to Google's network through a highly available, low latency connection.
    - Anthos service mesh connects with Istio Open Source service mesh via cloud interconnect.
    - Stackdriver for monitorning, alerting etc for hybrid network.
    - Anthos configuration Management provides a single source of truth for your cluster configurations.
    - That source of truth is in a policy repo which is indeed a git repo.

- App Engine (PaaS)
  - Scale automatically.
  - Offers two environments:
    - Standard
    - Flexible
  - Standard is simpler
  - Free daily usage quota
  - Low util apps might be able to run at no charge.
  - SDK in different languages.