Skip to content

Instantly share code, notes, and snippets.

@mrbobbytables

mrbobbytables/networking.md

Last active May 1, 2018 13:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mrbobbytables/be396bc0ffe440edec9d25e2601c9fed to your computer and use it in GitHub Desktop.
Save mrbobbytables/be396bc0ffe440edec9d25e2601c9fed to your computer and use it in GitHub Desktop.
Download ZIP
Networking
Raw
networking.md

missed first part due to laptop being dumb

Networking

pod "ready++" allow external systems (load-balancers, policies etc) to participate in rolling updates. External system decorates pod with ready status

Ingress

  • lowest common denominator api
  • very limited with with current configuration
  • annotation model of markup limits portability
  • ingress survey reports:
  • people want portability
  • everyone uses non-portable features..
  • 2018 L7 requirements are dramatically higher than what they were and many vendors don't support that level of functionality.
  • Possible Solution? Routes
    • openshift uses routes
    • heptio prototyping routes currently
  • All things considered, requirements are driving it closer and closer to istio
    • Possibility, poach some of the ideas and add them to kubernetes native.

Topology and node-local services

  • clear need for node-local network and service discovery
    • e.g. "I want to talk to the logging daemon on my current host"
  • These types of services should not be a service proper.

multi-network

certain scenarios demand multi-network

  • example: NFV
  • Problem: pods were not designed with multiple interfaces
    • Need to ensure that old mistakes are not repeated
    • developers don't want to think about routing and subnets etc, they just want it to work

net plugins vs device plugins

  • gpu that is also an infiniband device
  • causes problems because network and device are very different with verbs etc
  • problems encountered with having to schedule devices and network together at the same time.
    • "I want a gpu on this host that has a gpu attached and I want it to be the same deviec"
    • PoC available to make this work, but its rough and a problem right now.
  • difficult to work around, no right answer yet

net plugins, grpg, services

  • tighter coupling between net plugins and kube-proxy might be useful
  • grpc is awesome for plugins, why not use a grpc network plugin
  • pass services to network plugin to bypass kube-proxy, give more awareness to the network plugin and enable more functionality.

ipv6

  • beta now, NO support for dual-stack (v4/v6 at the same time)
  • need deeper changes like multiple pod ips, a problem as it impacts the pod API

service v3

  • services + endpoints "grew organically"

  • kind of a grab bag of features

  • need to start segmenting the "core" api group

    • move core apis to their own apigroup.

  • opportunity to rethink and refactor

    • endpoints -> endpoint
    • split the grouping construct from the "gazintas"
      • virtualIP, network, dns name moves into the service
    • EOL troublesome features
      • port remapping

DNS Reboot

  • kubernetes abuses DNS

  • Screwed up the schema for DNS and doesn't reflect the schema of the system

  • Have a few ideas on migrating to a new better DNS schema over a few years current issues with schema:

  • problem: its possible to write queries in dns that write over names

    • create a namespace called "com" and an app named "google" and it'll cause a problem

  • "svc" is an artifact and should not be a part of dns

  • issues with certain underlying libraries

  • "Enlighted DNS server"

    • Smart proxies on behalf of pods that do the searching and become a "better" dns

  • External DNS

  • Creates DNS entries in external system (route53)

  • Currently in incubator, not sure on status, possibly might move out of incubator, but unsure on path forward

Questions:

  • Feel like things are going in a circle on net/device plugins

    • Resource management on device/net plugin, feels like things are going in a spiral, but progress is being made, it is a very difficult problem and hard to keep all design points tracked. Trying to come to consensus on it all.

  • Would CoreDNS be the best place for the plugins and other modes for DNS proxy etc.

    • Possibly, seems extensible and likely the right choice. DNS itself is still a problem, couple other solutions would be do something like use UDP to communicate with DNS on localhost and TCP to a DNS server not on host. Lots of possibilities that can be solved with tools that are already controlled.

  • With the issues talked about earlier, why can't istio be integrated natively?

    • istio is still a little green and has its own issues, its an external project. Eventually its likely everyone will run something like it.

  • Thoughts on k8s v2?

    • Things will not just be turned off, things must be phased out and over the course of years, especially for services which have been core for some time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment