OpenShift Notes

OpenShift-Notes.md

Getting access

• Use the Openshift Client to connect to instances
  • oc login -u $USERNAME -p $PASSWORD
• Can also use the REST Api
  • Needs a bearer token, login with oc login then run oc whoami -t to get the token

TOKEN=`oc whoami -t`
curl --insecure -H "Authorization: Bearer $TOKEN" https://$TARGET:8443/oapi/v1

Project Setup

• Project resources are auto-namespaced to prevent conflicts in Kubernetes
• List projects: oc get projects

Creating a project

• Web UI

• CLI

Users and access

• List users: oc get users
  • The anypassword identity provider means the user can be access with an arbitray password
• Default system accounts can't be accessed in the WebUI and you'll need to access or create a cluster admin for such purposes.
  • oc adm policy add-cluster-role-to-user cluster-admin $USERNAME

Default Roles

• Regular
  • Developers and normal users.
  • Login is the userid.
• System
  • OpenShift Admins users for interacting with the infrastructure
  • Usernames are prefixed with system:
    • Default accounts are system:admin and system:master
• Service
  • Used for enabling communication between services in an application ie. Account for web server to access a database
  • Usernames are preficed with system:serviceaccount:
• A built in OAUTH server is generated and applies a Deny All identify provder is user.
  • Admins will need to create user accounts
    • Config File: /etc/openshift/master/master-config.yaml

Builds & Deployments

• Uses source code from a SCM ie, GitHub, GitLab, BitBucket, etc.
  • Once a repo is added to a project, an build job will be added for that repo/application
  • Running the build job will trigger a source download and build into a Docker image.
  • The image is the pushed to the built-in docker registry
  • Creates deployment to deploy the application to the k8s cluster
    • This is similar a K8s deployment, but OpenShift uses a DeploymentConfig object instead of the K8s Deployment object.

Builds

• Builds are configured with YAML files
• Simple way to boot strap is take the S2I YAML and customize it.
• Can trigger builds through web hooks, WebUI, or CLI command oc start-build ...
• Can configure environment variables and secrets similar to Jenkins

Build Strategies

1. Docker Build - Create a DockerFile with instructions to build the application
2. Souce-To-Image - Framework to convert source code into reusable Docker images
3. Custom Build - Used for buildining indidivual Gems, JARs, Python Packages, etc..

Image Streams

• Useful for when a build pulls a Docker image from a remote repo ie DockerHub.
• The docker image URL is abstracted as a pointer to image names referenced in build files
  • ie. docker.io/centos/python-27:latest is mapped to simply python-27:latest
    • This will refernce the image ID instead of the image tag to prevent breaking updates in the tagged image version

Deployments

• Similar to deployments in K8S
  • K8s uses "Pods" containing one or more docker containers

• Common Deployment stategies are:
• Recreate - kill are instances and redploy all instances
• Rolling - Default. Destroy and rebuild instance squentially.
  • oc rollout latest $DEPLOYMENT

Networks, Services, Routes, and Scaling

Networks (internal)

• K8s Netowrking recap:
  • Each pod on a worker node gets an IP address
• OpenShift Software Defined Network creates a virtual networks that spans across multiple nodes called an Overlay Network
  • Uses Open vSwitch standard to handle VLAN tagging, Trunking, LACP, Port Mirroring, etc..
  • Default network is 10.128.0.0/14
  • Each nodes get a subnet ie 10.128.2.0/23
    • All pods get a unique IP in the node's assigned subnet
• Can check IPs of pods using oc get pods -o wide
• Since IPs are ephemeral, a builtin SkyDNS server allows using pod/service names to connect to other pods.
• SDN plugins are available, the default one used in ovs-subnet which provides connection between all pods in the cluster
• The ovs-multitenant plugin can be used to assign each project a unique virtual network ID to be segmented from other projects.

Services & Routes (internal/external)

Services

• Services connect different applictions/pods
  • Stored in service-config.yaml

• Each service gets its own Cluster IP and DNS entries
• Services are linked to pods using selectors
  • ie deploymentconfig=nodejs-frontend and the appropriate port

Routes

• Routes expose services to external users as hostname.
  • Load Balancing
    • Source (Default method) - Uses the source IP to send traffic to same backend server for the session duration.
    • Round Robin
    • Least Connections
  • Security
    • Can add TLS cert for encrypted connections
    • Can also prevent HTTP connections or redirct to HTTPs
  • Split Traffic
    • Can be used for A/B testing

Scaling

• Easy as adjusting the number of pods to have online
• May need to adjust the load balancing method

Storage, Templates, and Catalog

Storage

• Attach persistant volumes to allow data to persist between containers
• Access Modes
  • Single User - Allows volume to be mounted read/write by a single node
  • Shared Access - Read/write access to multiple nodes
  • Read Only - Read only use by multple nodes
• Add as a volume to the deployment config

Templates and Catalog

• Can often find these on Github for services you don't have by default
• Templates are configurations of contains, servcies, routes, and paramenters to make a cohesive application
• Define template as a YAML file and deploy with oc create -f $TEMPLATE_FILE
  • Can get an example by exporting a template, ie oc export service db
• Normal users cannot create templates in the default namespace, you have to specify a project namesapce
• Can parameterize templates to allow for customizing deploymenmts

Micoservices example

• Voting frontend collects votes
• Commits votes to redis
• A .Net worker updates the persistant Postgres DB
• The Results App reads the DB to show the totals

Pentesting notes

Remote Access

• Remote terminal: oc rsh $POD

Info Gathering

• Check the Deployment environment variables for potential sensitive data
  • Passwords should be stored as secrets, but devs can be lazy
• Check the secrets to see if they can be leaked

Adding moar stuff