Skip to content

Instantly share code, notes, and snippets.

@roycaihw

roycaihw/relnotes-1.9.md

Created November 29, 2017 21:29
Show Gist options
  • Save roycaihw/62279aa12a6d7cbcb9a9d24dd4b0bbad to your computer and use it in GitHub Desktop.
Save roycaihw/62279aa12a6d7cbcb9a9d24dd4b0bbad to your computer and use it in GitHub Desktop.
Download ZIP
example changelog since 1.8.0
Raw
relnotes-1.9.md

master

Documentation & Examples

Changelog since v1.8.0

Action Required

Api-Machinery

NullArea

  • NullIssue

    • Graduate the admission and admissionregistration (webhook part) API to v1beta1 (#56004, @caesarxuchao)
    • action required: Deprecated flags --portal-net and service-node-ports of kube-apiserver are removed. (#52547, @xiangpengzhao)

Auth

NullArea

  • unable to deploy privileged pod after 1.8 upgrade unless I set allowPrivilegeEscalation true (#53437)

    • PodSecurityPolicy: Fixes a compatibility issue that caused policies that previously allowed privileged pods to start forbidding them, due to an incorrect default value for allowPrivilegeEscalation. PodSecurityPolicy objects defined using a 1.8.0 client or server that intended to set allowPrivilegeEscalation to false must be reapplied after upgrading to 1.8.1. (#53443, @liggitt)

  • NullIssue

    • RBAC objects are now stored in etcd in v1 format. After completing an upgrade to 1.9, RBAC objects (Roles, RoleBindings, ClusterRoles, ClusterRoleBindings) should be migrated to ensure all persisted objects are written in v1 format, prior to v1alpha1 support being removed in a future release. (#52950, @liggitt)

NullSig

NullArea

  • NullIssue

    • update podtolerations admission to mutate and validate separately (#55251, @deads2k)
    • kubeadm join: Error out if CA pinning isn't used or opted out of (#55468, @yuexiao-wang)

Storage

NullArea

  • Volume topology aware scheduling binding changes (#54435)

    • Topology aware volume scheduler and PV controller changes (#55039, @msau42)

  • NullIssue

    • action required: The storage.k8s.io/v1beta1 API and volume.beta.kubernetes.io/storage-class annotation are deprecated. They will be removed in a future release. Please use v1 API and field v1.PersistentVolumeClaim.Spec.StorageClassName/v1.PersistentVolume.Spec.StorageClassName instead. (#53580, @xiangpengzhao)

Other notable changes

Api-Machinery

Admission-Control

  • NullIssue

  • Authentication for webhook admission to heterogenous authentication domains (#54404)

    • generic webhook admission now takes a config file which describes how to authenticate to webhook servers (#54414, @deads2k)

  • ExternalAdmissionHookConfiguration cannot choose URL (#53826)

    • admission webhook registration now allows URL paths (#54145, @deads2k)

Api

  • Eliminate Phase and Conditions from the API (#7856)

    • StatefulSet status now has support for conditions, making it consistent with other core controllers in v1 (#55268, @foxish)

Api & Apiserver

  • Reported subresource discovery is incorrect (#54684)

    • Fixes discovery information for scale subresources in the apps API group (#54683, @liggitt)

Apiserver

  • NullIssue

    • Fix a bug that prevents client-go metrics from being registered in prometheus in multiple components. (#53434, @crassirostris)

Audit

  • Shutdown http handlers before shutting down audit backend (#50781)

    • Implement graceful shutdown of the kube-apiserver by waiting for open connections to finish before exiting. Moreover, the audit backend will stop dropping events on shutdown. (#53695, @hzxuzhonghu)

  • [audit] Always retry sending to webhook (#52909)

Custom-Resources

  • Getting CRD Validation to Beta (#53829)

    • Promote validation for custom resources defined through CRD to beta (#54647, @colemickens)

  • sample-controller example repository (#52752)

  • Unable to use a fieldSelector with custom resources (#51046)

    • Custom resources served through CustomResourceDefinition now support field selectors for metadata.name and metadata.namespace. (#53345, @ncdc)

  • CRD and TPR doesn't support watching one single instance (#49424)

    • Custom resources served through CustomResourceDefinition now support field selectors for metadata.name and metadata.namespace. (#53345, @ncdc)

Etcd

Hw-Accelerators

  • NullIssue

Ipv6

  • NullIssue

      • Fix handling of IPv6 URLs in NO_PROXY. (#53898, @kad)

NullArea

  • should prevent the deletion of a PVC that is referenced by an active pod (#45143)

    • PVC Finalizing Controller is introduced in order to prevent deletion of a PVC that is being used by a pod. (#55824, @pospispa)

  • kube-apiserver "no --service-cluster-ip-range specified" and "Defaulting to 10.0.0.0/24". (#52695)

    • Fixed a bug which is causes kube-apiserver to not run without specifying service-cluster-ip-range (#52870, @jennybuckley)

  • kubectl attach: client-go does not respect CIDRs in NO_PROXY (#54407)

    • API machinery's httpstream/spdy calls now support CIDR notation for NO_PROXY (#54413, @kad)

  • Enhance the codegen script within the staging sample apiserver to work with multiple groups and versions (#48714)

    • Add generate-groups.sh and generate-internal-groups.sh to k8s.io/code-generator to easily run generators against CRD or User API Server types. (#52186, @sttts)

  • NullIssue

    • Admission response alt (#55829, @cheftako)
    • Google KMS integration was removed from in-tree in favor of a out-of-process extension point that will be used for all KMS providers. (#54759, @sakshamsharma)
    • DaemonSet, Deployment, ReplicaSet, and StatefulSet have been promoted to GA and are available in the apps/v1 group version. (#53679, @kow3ns)
    • kubectl get will by default fetch large lists of resources in chunks of up to 500 items rather than requesting all resources up front from the server. This reduces the perceived latency of managing large clusters since the server returns the first set of results to the client much more quickly. A new flag --chunk-size=SIZE may be used to alter the number of items or disable this feature when 0 is passed. This is a beta feature. (#53768, @smarterclayton)
    • apiserver: --etcd-quorum-read now defaults to true, to ensure correct operation with HA etcd clusters (#53717, @liggitt)
    • API chunking via the limit and continue request parameters is promoted to beta in this release. Client libraries using the Informer or ListWatch types will automatically opt in to chunking. (#52949, @smarterclayton)
    • Add events.k8s.io api group with v1beta1 API containing redesigned Event type. (#49112, @gmarek)
    • kubectl apply use openapi to calculate diff be default. It will fall back to use baked-in types when openapi is not available. (#51321, @mengqiy)
    • The GenericAdmissionWebhook is renamed as ValidatingAdmissionWebhook. Please update you apiserver configuration file to use the new name to pass to the apiserver's --admission-control flag. (#55988, @caesarxuchao)
    • The apiserver sends external versioned object to the admission webhooks now. Please update the webhooks to expect admissionReview.spec.object.raw to be serialized external versions of objects. (#55127, @caesarxuchao)
    • ReplicationController now shares its underlying controller implementation with ReplicaSet to reduce the maintenance burden going forward. However, they are still separate resources and there should be no externally visible effects from this change. (#49429, @enisoc)

  • [apps/v1] Change DefaultGarbageCollectionPolicy for workload controllers (#55027)

    • change DefaultGarbageCollectionPolicy to DeleteDependents for workloads controllers (#55148, @dixudx)

  • Collect metrics on admission rejections (#55030)

    • Metrics have been added for monitoring admission plugins, including the new dynamic (webhook-based) ones. (#55183, @jpbetz)

  • controller-manager crash loops if gc controller doesn't have access to extension apis (#55022)

    • API discovery failures no longer crash the kube controller manager via the garbage collector. (#55259, @ironcladlou)

  • conversion-gen --extra-peer-dirs references k8s.io/kubernetes types (#54301)

    • In conversion-gen removed Kubernetes core API from default extra-peer-dirs. (#54394, @sttts)

  • "kubectl explain" should be able to explain "apiservices" and "customresourcedefinition" (#49465)

    • kubectl explain now uses openapi rather than swagger 1.2. (#53228, @apelisse)

  • client-gen tag shortcomings when newline is omitted (#53893)

    • Allow code-generator tags in the 2nd closest comment block and directly above a statement. (#55233, @sttts)

  • PersistentVolumeSource should be read-only (#54562)

    • Validate that PersistentVolumeSource is not changed during PV Update (#54761, @ianchakeres)

  • apiserver proxy feature does not rewrite Location header on redirects (#51790)

    • Apiserver proxy rewrites URL when service returns absolute path with request's host. (#52556, @roycaihw)

  • Update gRPC library to pick up data race fix (#53124)

  • kubectl set commands on ReplicaSet and DaemonSet occasionally return version registration errors (#53040)

    • Fixes an issue with kubectl set commands encountering conversion errors for ReplicaSet and DaemonSet objects (#53158, @liggitt)

  • apiserver uses wrong CommonName to verify service certificates for aggregated API Server when External admission controller is enabled (#56385)

    • Fixes server name verification of aggregated API servers and webhook admission endpoints (#56415, @liggitt)

Apps

Api

  • Eliminate Phase and Conditions from the API (#7856)

    • StatefulSet status now has support for conditions, making it consistent with other core controllers in v1 (#55268, @foxish)

Batch & Workload-Api/Job

  • Implement scale endpoint for jobs (#38756)

    • RBAC PolicyRules now allow resource=*/<subresource> to cover any-resource/<subresource>. For example, */scale covers replicationcontroller/scale. (#53722, @deads2k)

NullArea

  • Remove CreatedByAnnotation in v1.9, in favor of ControllerRef (#50720)

    • The kubernetes.io/created-by annotation is no longer added to controller-created objects. Use the metadata.ownerReferences item that has controller set to true to determine which controller, if any, owns an object. (#54445, @crimsonfaith91)

  • kubectl scale implementation for core workload controllers (#49504)

    • RBAC PolicyRules now allow resource=*/<subresource> to cover any-resource/<subresource>. For example, */scale covers replicationcontroller/scale. (#53722, @deads2k)

  • [apps/v1] Change DefaultGarbageCollectionPolicy for workload controllers (#55027)

    • change DefaultGarbageCollectionPolicy to DeleteDependents for workloads controllers (#55148, @dixudx)

  • NullIssue

    • StatefulSet controller will create a label for each Pod in a StatefulSet. The label is named statefulset.kubernetes.io/pod-name and it is equal to the name of the Pod. This allows users to create a Service per Pod to expose a connection to individual Pods. (#55329, @kow3ns)
    • ReplicationController now shares its underlying controller implementation with ReplicaSet to reduce the maintenance burden going forward. However, they are still separate resources and there should be no externally visible effects from this change. (#49429, @enisoc)
    • DaemonSet status now has a new field named "conditions", making it consistent with other workloads controllers. (#55272, @janetkuo)
    • DaemonSet, Deployment, ReplicaSet, and StatefulSet have been promoted to GA and are available in the apps/v1 group version. (#53679, @kow3ns)
    • Add API version apps/v1, and bump DaemonSet to apps/v1 (#53278, @janetkuo)

Architecture

Api

  • Eliminate Phase and Conditions from the API (#7856)

    • StatefulSet status now has support for conditions, making it consistent with other core controllers in v1 (#55268, @foxish)

NullArea

  • NullIssue

    • Google KMS integration was removed from in-tree in favor of a out-of-process extension point that will be used for all KMS providers. (#54759, @sakshamsharma)

Auth

Admission-Control

  • Authentication for webhook admission to heterogenous authentication domains (#54404)

    • generic webhook admission now takes a config file which describes how to authenticate to webhook servers (#54414, @deads2k)

Audit

  • [audit] Figure out timestamps in event objects (#52160)

    • add RequestReceivedTimestamp and StageTimestamp to audit event (#52981, @CaoShuFeng)

  • [audit] Always retry sending to webhook (#52909)

Kubeadm

  • CertificateManager blocks kubelet start if auto-approval is not enabled (#53237)

    • kubeadm allows the kubelets in the cluster to automatically renew their client certificates (#53252, @kad)

Kubeadm & Kubelet

  • TLS-bootstrapped kubelet loses client certs after reboot, node stays on NotReady status (#53288)

    • kubelet --cert-dir now defaults to /var/lib/kubelet/pki, in order to ensure bootstrapped and rotated certificates persist beyond a reboot. resolves an issue in kubeadm with false-positive /var/lib/kubelet is not empty message during pre-flight checks (#53317, @liggitt)

NullArea

  • NullIssue

    • Implement kubelet side file system resizing. Also implement GCE PD resizing (#55815, @gnufied)
    • kubeadm: Add an experimental mode to deploy CoreDNS instead of KubeDNS (#52501, @rajansandeep)
    • Google KMS integration was removed from in-tree in favor of a out-of-process extension point that will be used for all KMS providers. (#54759, @sakshamsharma)
    • Pod Security Policy can now manage access to specific FlexVolume drivers (#53179, @wanghaoran1988)
    • The RBAC bootstrapping policy now allows authenticated users to create selfsubjectrulesreviews. (#56095, @ericchiang)
    • Defaulting of controller-manager options for --cluster-signing-cert-file and --cluster-signing-key-file is deprecated and will be removed in a later release. (#54495, @mikedanese)
    • RBAC ClusterRoles can now select other roles to aggregate (#54005, @deads2k)
    • Audit policy files without apiVersion and kind are treated as invalid. (#54267, @ericchiang)
    • Resolves forbidden error when accessing replicasets and daemonsets via the apps API group (#54309, @liggitt)
    • RBAC: The default admin and edit roles now include read/write permissions and the view role includes read permissions on poddisruptionbudget.policy resources. (#52654, @liggitt)

  • [PodSecurityPolicy] Optimize getMatchingPolicies (#55521)

    • Improved PodSecurityPolicy admission latency, but validation errors are no longer limited to only errors from authorized policies. (#55643, @tallclair)

  • OIDC username prefix option is not working (#56169)

    • kube-apiserver: fixed --oidc-username-prefix and --oidc-group-prefix flags which previously weren't correctly enabled (#56175, @ericchiang)

  • RFE: Bootstrap Checkpointing - Modify manifest behavior slightly for self hosting. (#49236)

  • Proposal: support unequivocal DENY in union authorizer (#51862)

    • Add support for the webhook authorizer to make a Deny decision that short-circuits the union authorizer and immediately returns Deny. (#53273, @mikedanese)

  • Split PSP defaulting and validation (#36184)

    • PodSecurityPolicy: when multiple policies allow a submitted pod, priority is given to ones which do not require any fields in the pod spec to be defaulted. If the pod must be defaulted, the first policy (ordered by name) that allows the pod is used. (#52849, @liggitt)

  • Certificate Signing Request cleaner to GC CSRs (#51550)

  • Reconciliation adds duplicated subjects on server start (#53296)

    • Fixes an issue with RBAC reconciliation that could cause duplicated subjects in some bootstrapped rolebindings on each restart of the API server. (#53239, @enj)

Platform/Gce

  • NullIssue

    • Add support for PodSecurityPolicy on GCE: ENABLE_POD_SECURITY_POLICY=true enables the admission controller, and installs policies for default addons. (#52367, @tallclair)

Security

  • Bring PodSecurityPolicy to usable state (#23217)

    • PodSecurityPolicy: when multiple policies allow a submitted pod, priority is given to ones which do not require any fields in the pod spec to be defaulted. If the pod must be defaulted, the first policy (ordered by name) that allows the pod is used. (#52849, @liggitt)

Autoscaling

NullArea

  • HPA is still using replicationcontrollers.extensions/scale (#38810)

    • Introduces a polymorphic scale client, allowing HorizontalPodAutoscalers to properly function on scalable resources in any API group. (#53743, @DirectXMan12)
    • RBAC PolicyRules now allow resource=*/<subresource> to cover any-resource/<subresource>. For example, */scale covers replicationcontroller/scale. (#53722, @deads2k)

  • NullIssue

  • kubectl scale implementation for core workload controllers (#49504)

    • RBAC PolicyRules now allow resource=*/<subresource> to cover any-resource/<subresource>. For example, */scale covers replicationcontroller/scale. (#53722, @deads2k)

  • Update HPA tolerance to be a flag (#18155)

  • HPA scaling above spec.maxReplicas (#53670)

    • Address a bug which allowed the horizontal pod autoscaler to allocate desiredReplicas > maxReplicas in certain instances. (#53690, @mattjmcnaughton)

Aws

NullArea

  • Taint a AWS node if a volume is stuck in "attaching" state for too long (#55502)

    • AWS: Apply taint to a node if volumes being attached to it are stuck in attaching state (#55558, @gnufied)

  • AWS makes high number of redundant AttachVolume and DeleteVolume calls (#55014)

    • Check for available volume before attach/delete operation in EBS (#55008, @gnufied)

  • AWS error messages printed on 2 lines (#49813)

    • avoid newline "\n" in the error to break log msg to 2 lines (#49826, @dixudx)

  • Detach is broken from stopped nodes in AWS (#55892)

  • NullIssue

    • Add support for resizing EBS disks (#56118, @gnufied)
    • It is now possible to override the healthcheck parameters for AWS ELBs via annotations on the corresponding service. The new annotations are healthy-threshold, unhealthy-threshold, timeout, interval (all prefixed with service.beta.kubernetes.io/aws-load-balancer-healthcheck-) (#56024, @dimpavloff)
    • Support AWS ECR credentials in China (#50108, @zzq889)

  • Enable AWS Network Load Balancer for Services of type LoadBalancer (#52173)

  • Adding tag annotations on service manifest for ELB does not trigger update on AWS (#54642)

Azure

NullArea

  • Azure data disk should provision storage account on on-demand (#50883)

  • [Azure] Support setting the DNS name label for public IPs created by ingress controllers (#44775)

    • New service annotation "service.beta.kubernetes.io/azure-dns-label-name" to set Azure DNS label name for public IP (#47849, @tomerf)

  • there are lots of warning message due to GetMountRefs func in windows (#54670)

    • fix warning messages due to GetMountRefs func not implemented in windows (#52401, @andyzhangx)

  • Panic in azure_dd/azure_mounter.go when syncing pod (#54149)

  • azure_dd: managed disks don't pass "FormatAndMount" (#50150)

    • fix azure disk mount failure on coreos and some other distros (#54334, @andyzhangx)

  • Azure disk: storage class should support the sku if the storage accout support it (#55774)

  • wrong controller-master detection (#54570)

  • Azure disk dose not work as expected (#55776)

  • NullIssue

  • there is azure file mount limit issue on windows due to using drive letter (#54668)

    • fix azure file mount limit issue on windows due to using drive letter (#53629, @andyzhangx)

  • Azure loadbalancer should reconcile security groups properly. (not just by name, but also by other properties) (#55733)

    • Kubernetes update Azure nsg rules based on not just difference in Name, but also in Protocol, SourcePortRange, DestinationPortRange, SourceAddressPrefix, DestinationAddressPrefix, Access, and Direction. (#55752, @kevinkim9264)

Platform/Azure

  • azure_file volumes should allow setting of dir_mode and file_mode (#37005)

Cli

Federation

  • NullIssue

    • Development of Kubernetes Federation has moved to github.com/kubernetes/federation. This move out of tree also means that Federation will begin releasing separately from Kubernetes. The impact of this is Federation-specific behavior will no longer be included in kubectl, kubefed will no longer be released as part of Kubernetes, and the Federation servers will no longer be included in the hyperkube binary and image. (#53816, @marun)

Kubectl

  • NullIssue

  • Kubectl: Replace usages of swagger with open API (#44589)

    • Kubectl: Remove swagger 1.2 validation. Also removes options --use-openapi and --schema-cache-dir as these are no longer needed. (#53232, @apelisse)

  • Unable to use kubectl get with a fieldSelector (#14129)

  • kubectl scale should use the scale subresource (#29698)

    • RBAC PolicyRules now allow resource=*/<subresource> to cover any-resource/<subresource>. For example, */scale covers replicationcontroller/scale. (#53722, @deads2k)

NullArea

  • NullIssue

    • Added --dry-run option to kubectl drain (#52440, @juanvallejo)
    • outputs <none> for columns specified by -o custom-columns but not found in object (#51750, @jianhuiz)
    • kubectl apply use openapi to calculate diff be default. It will fall back to use baked-in types when openapi is not available. (#51321, @mengqiy)
    • kubectl create pdb will no longer set the min-available field by default. (#53047, @yuexiao-wang)
    • DaemonSet, Deployment, ReplicaSet, and StatefulSet have been promoted to GA and are available in the apps/v1 group version. (#53679, @kow3ns)
    • kubectl get will by default fetch large lists of resources in chunks of up to 500 items rather than requesting all resources up front from the server. This reduces the perceived latency of managing large clusters since the server returns the first set of results to the client much more quickly. A new flag --chunk-size=SIZE may be used to alter the number of items or disable this feature when 0 is passed. This is a beta feature. (#53768, @smarterclayton)
    • add --raw to kubectl create to POST using the normal transport (#54245, @deads2k)
    • "kubectl cp" updated to honor destination names (#51215, @juanvallejo)

  • Add create priorityclass sub command (#54857)

  • kubectl scale implementation for core workload controllers (#49504)

    • RBAC PolicyRules now allow resource=*/<subresource> to cover any-resource/<subresource>. For example, */scale covers replicationcontroller/scale. (#53722, @deads2k)

  • "kubectl explain" should be able to explain "apiservices" and "customresourcedefinition" (#49465)

    • kubectl explain now uses openapi rather than swagger 1.2. (#53228, @apelisse)

  • kubectl set commands on ReplicaSet and DaemonSet occasionally return version registration errors (#53040)

    • Fixes an issue with kubectl set commands encountering conversion errors for ReplicaSet and DaemonSet objects (#53158, @liggitt)

Usability

  • top pod and top node output is unstable ordered (#53513)

    • kubectl top pod and node should sort by namespace / name so that results don't jump around. (#53560, @dixudx)

Cluster-Lifecycle

Apiserver & Cloudprovider & Platform/Gce

  • NullIssue

    • kube-apiserver: --ssh-user and --ssh-keyfile are now deprecated and will be removed in a future release. Users of SSH tunnel functionality used in Google Container Engine for the Master -> Cluster communication should plan to transition to alternate methods for bridging master and node networks. (#54433, @dims)

Cloudprovider

  • NullIssue

  • Expose concurrent-service-syncs flag on the CCM like it is for the KCM (#55560)

    • Add the concurrent service sync flag to the Cloud Controller Manager to allow changing the number of workers. (--concurrent-service-syncs) (#55561, @jhorwit2)

  • cloud controller manager does not support configmap resource locks (#55124)

  • Remove --cloud-provider=auto-detect (#50986)

    • kubelet's --cloud-provider flag no longer defaults to "auto-detect". If you want cloud-provider support in kubelet, you must set a specific cloud-provider explicitly. (#53573, @dims)

Etcd

Hw-Accelerators

  • NullIssue

    • GCE nodes with NVIDIA GPUs attached now expose nvidia.com/gpu as a resource instead of alpha.kubernetes.io/nvidia-gpu. (#54826, @mindprince)

Ipv6

  • Add kubeadm config for setting kube-proxy BindAddress (#50927)

    • Adds kubeadm support for using ComponentConfig for the kube-proxy (#55972, @rpothier)

  • Use CNI v0.6.x in Kubernetes v1.9.0 (#49480)

Kubeadm

  • CertificateManager blocks kubelet start if auto-approval is not enabled (#53237)

    • kubeadm allows the kubelets in the cluster to automatically renew their client certificates (#53252, @kad)

  • NullIssue

    • kubeadm health checks can also be skipped with --ignore-checks-errors (#56130, @anguslees)
    • Implement individual control for kubeadm preflight checks (#56072, @kad)
    • kubeadm now produces error during preflight checks if swap is enabled. Users, who can setup kubelet to run in unsupported environment with enabled swap, will be able to skip that preflight check. (#55399, @kad)
      • kubeadm will produce error if kubelet too new for control plane (#54868, @kad)

  • kubeadm 1.8.0 init fails with "/var/lib/kubelet is not empty" (#53356)

    • kubelet --cert-dir now defaults to /var/lib/kubelet/pki, in order to ensure bootstrapped and rotated certificates persist beyond a reboot. resolves an issue in kubeadm with false-positive /var/lib/kubelet is not empty message during pre-flight checks (#53317, @liggitt)

Kubeadm & Kubelet

  • TLS-bootstrapped kubelet loses client certs after reboot, node stays on NotReady status (#53288)

    • kubelet --cert-dir now defaults to /var/lib/kubelet/pki, in order to ensure bootstrapped and rotated certificates persist beyond a reboot. resolves an issue in kubeadm with false-positive /var/lib/kubelet is not empty message during pre-flight checks (#53317, @liggitt)

Kubelet-Api

  • Specifying feature gates as a string of key-value pairs in ComponentConfig structures is awkward (#53024)

    • The Kubelet's feature gates are now specified as a map when provided via a JSON or YAML KubeletConfiguration, rather than as a string of key-value pairs. (#53025, @mtaufen)

NullArea

  • NullIssue

      • kubeadm will warn users if access to IP ranges for Pods or Services will be done via HTTP proxy. (#52792, @kad)
    • kubeadm: added --print-join-command flag for kubeadm token create. (#56185, @mattmoyer)
    • Adding etcd version display to kubeadm upgrade plan subcommand (#56156, @sbezverk)
    • Adds to kubeadm upgrade apply, a new --etcd-upgrade keyword. When this keyword is specified, etcd's static pod gets upgraded to the etcd version officially recommended for a target kubernetes release. (#55010, @sbezverk)
    • Kubeadm now supports for Kubelet Dynamic Configuration. (#55803, @xiangpengzhao)
    • Base images bumped to Debian Stretch (9) (#52744, @rphillips)
    • kubeadm: Add an experimental mode to deploy CoreDNS instead of KubeDNS (#52501, @rajansandeep)
    • kubeadm: Add CoreDNS support for kubeadm "upgrade" and "alpha phases addons". (#55952, @rajansandeep)
    • kubeadm init: fix a bug that prevented the --token-ttl flag and tokenTTL configuration value from working as expected for infinite (0) values. (#54640, @mattmoyer)
    • Feature gates now check minimum versions (#54539, @jamiehannaford)
    • Load kernel modules automatically inside a kube-proxy pod (#52003, @vfreex)

  • RFE: Bootstrap Checkpointing - Modify manifest behavior slightly for self hosting. (#49236)

  • Refactor kube-scheduler configuration (#52428)

    • The kube-scheduler command now supports a --config flag which is the location of a file containing a serialized scheduler configuration. Most other kube-scheduler flags are now deprecated. (#52562, @ironcladlou)

  • Creation of gitRepo volume is broken in 1.8.0+ (#54129)

    • Add openssh-client back into the hyperkube image. This allows the gitRepo volume plugin to work properly. (#54250, @ixdy)

  • Update to Go 1.9 (#49484)

Contributor-Experience

NullArea

  • NullIssue

    • kubectl apply use openapi to calculate diff be default. It will fall back to use baked-in types when openapi is not available. (#51321, @mengqiy)

Federation (Deprecated - Do Not Use)

NullArea

  • Enable kubefed init support nodeSelector (#50716)

Gcp

Apiserver & Cloudprovider & Platform/Gce

  • NullIssue

    • kube-apiserver: --ssh-user and --ssh-keyfile are now deprecated and will be removed in a future release. Users of SSH tunnel functionality used in Google Container Engine for the Master -> Cluster communication should plan to transition to alternate methods for bridging master and node networks. (#54433, @dims)

Platform/Gce

  • GCE should allow users to configure with what service account their nodes are created (#53603)

    • Allow GCE users to configure the service account made available on their nodes (#52868, @ihmccreery)

Instrumentation

Audit

  • NullIssue

    • Adjust batching audit webhook default parameters: increase queue size, batch size, and initial backoff. Add throttling to the batching audit webhook. Default rate limit is 10 QPS. (#53417, @crassirostris)

  • [audit] Always retry sending to webhook (#52909)

Hw-Accelerators

  • NullIssue

    • Kubelet now exposes metrics for NVIDIA GPUs attached to the containers. (#55188, @mindprince)

Logging

  • NullIssue

    • [fluentd-gcp addon] Fixes fluentd deployment on GCP when custom resources are set. (#55950, @crassirostris)
    • [fluentd-gcp addon] Fluentd now runs in its own network, not in the host one. (#54395, @crassirostris)

NullArea

  • NullIssue

    • Fix a typo in prometheus-to-sd configuration, that drops some stackdriver metrics. (#56473, @loburm)
    • [fluentd-elasticsearch addon] Elasticsearch and Kibana are updated to version 5.6.4 (#55400, @mrahbar)
    • A new field is added to CRI container log format to support splitting a long log line into multiple lines. (#55922, @Random-Liu)
    • fluentd now supports CRI log format. (#54777, @Random-Liu)

  • Collect metrics on admission rejections (#55030)

    • Metrics have been added for monitoring admission plugins, including the new dynamic (webhook-based) ones. (#55183, @jpbetz)

  • Bring all prom-to-sd container to the same image version (#54583)

    • Reduce log noise produced by prometheus-to-sd, by bumping it to version 0.2.2. (#54635, @loburm)

Multicluster

Federation

  • NullIssue

    • Development of Kubernetes Federation has moved to github.com/kubernetes/federation. This move out of tree also means that Federation will begin releasing separately from Kubernetes. The impact of this is Federation-specific behavior will no longer be included in kubectl, kubefed will no longer be released as part of Kubernetes, and the Federation servers will no longer be included in the hyperkube binary and image. (#53816, @marun)

  • Enable kubefed init support ImagePullSecrets and imagePullPolicy (#50718)

    • support imagePullSecrets and imagePullPolicy in kubefed init (#50740, @dixudx)

  • Get clusters --show-labels does not work in a federation context (#53729)

    • update cluster printer to enable --show-labels (#53771, @dixudx)

Network

Controller-Manager

  • Pod in graceful termination should not be on the ready address list of related Endpoints objects (#54723)

    • Fix a bug where pod address is not removed from endpoints object while pod is in graceful termination. (#54828, @freehan)

Ipv6

Ipvs

  • Need to install ipset in debian-iptables docker image (#56116)

  • IPVS kube-proxy will flush all existing ipvs rules in its startup (#55857)

Ipvs & Kube-Proxy

  • Try ipset in kube-proxy (#54203)

  • Failed to access NodePort if kube-proxy running in ipvs mode (#53393)

Kube-Proxy

  • Handle nodes with iptables FORWARD DROP better (#39823)

    • Add iptables rules to allow Pod traffic even when default iptables policy is to reject. (#52569, @tmjd)

NullArea

  • Support annotations for AWS ELB Security Policies (#43744)

  • zero-value settings for kube-proxy being overwritten by default values (#50787)

    • Restored kube-proxy's support for 0 values for conntrack min, max, max per core, tcp close wait timeout, and tcp established timeout. (#55261, @ncdc)

  • Calico add-on: calico/node pod can take a long time to be restarted (#55013)

    • Disable the termination grace period for the calico/node add-on DaemonSet to reduce downtime during a rolling upgrade or deletion. (#55015, @fasaxc)

  • Add CoreDNS in kube-up (#56439)

  • NullIssue

    • Fixes bad conversion in host port chain name generating func which leads to some unreachable host ports. (#55153, @chenchun)
    • kubeadm: Add an experimental mode to deploy CoreDNS instead of KubeDNS (#52501, @rajansandeep)
    • Fix IPVS availability check (#51874, @vfreex)
    • Enhanced the network policy describer. (#46951, @aanm)
    • Load kernel modules automatically inside a kube-proxy pod (#52003, @vfreex)
    • Improve resilience by annotating kube-dns addon with podAntiAffinity to prefer scheduling on different nodes. (#52193, @StevenACoffman)
    • Add DNSConfig field to PodSpec and support "None" mode for DNSPolicy (Alpha). (#55848, @MrHohn)

  • kube-proxy: session affinity stops working when ESIPP=Local (#55429)

    • Fix session affinity issue with external load balancer traffic when ExternalTrafficPolicy=Local. (#55519, @MrHohn)

  • [kubelet] ignore keyword "options" define in /etc/resolv.conf, only look for nameserver and search (#42542)

    • Support copying "options" in resolv.conf into pod sandbox when dnsPolicy is Default (#54773, @phsiao)

  • Service controller retries on doNotRetry service update failure (#54183)

    • Fix for service controller so that it won't retry on doNotRetry service update failure. (#54184, @MrHohn)

  • apiserver proxy feature does not rewrite Location header on redirects (#51790)

    • Apiserver proxy rewrites URL when service returns absolute path with request's host. (#52556, @roycaihw)

Platform/Gce

  • NullIssue

  • GCE: ILB sync fails for legacy networks and auto networks with unusual subnet names (#53409)

    • GCE: Fixes ILB sync on legacy networks and auto networks with unique subnet names (#53410, @nicksardo)

  • GCE: Ignore resource not found errors when deleting LB resources (#53411)

    • GCE: Fix issue deleting internal load balancers when the firewall resource may not exist. (#53450, @nicksardo)

Node

Cloudprovider

  • Remove --cloud-provider=auto-detect (#50986)

    • kubelet's --cloud-provider flag no longer defaults to "auto-detect". If you want cloud-provider support in kubelet, you must set a specific cloud-provider explicitly. (#53573, @dims)

Hw-Accelerators

  • NullIssue

    • Expose single annotation/label via downward API (#55902, @yguo0905)
    • Kubelet now exposes metrics for NVIDIA GPUs attached to the containers. (#55188, @mindprince)
    • Device plugin Alpha API no longer supports returning artifacts per device as part of AllocateResponse. (#53031, @vishh)

Ipv6

Kubeadm

  • CertificateManager blocks kubelet start if auto-approval is not enabled (#53237)

    • kubeadm allows the kubelets in the cluster to automatically renew their client certificates (#53252, @kad)

Kubelet & Kubelet-Api

  • NullIssue

    • The EvictionHard, EvictionSoft, EvictionSoftGracePeriod, EvictionMinimumReclaim, SystemReserved, and KubeReserved fields in the KubeletConfiguration object (kubeletconfig/v1alpha1) are now of type map[string]string, which facilitates writing JSON and YAML files. (#54823, @mtaufen)
    • Relative paths in the Kubelet's local config files (--init-config-dir) will be resolved relative to the location of the containing files. (#55648, @mtaufen)
    • It is now possible to set multiple manifest url headers via the Kubelet's --manifest-url-header flag. Multiple headers for the same key will be added in the order provided. The ManifestURLHeader field in KubeletConfiguration object (kubeletconfig/v1alpha1) is now a map[string][]string, which facilitates writing JSON and YAML files. (#54643, @mtaufen)

Kubelet & Security

  • Dockershim doesn't consider seccomp profile root? (#55359)

    • Fix CRI localhost seccomp path in format localhost//profileRoot/profileName. (#55450, @feiskyer)

Kubelet-Api

  • NullIssue

    • The Kubelet's --enable-custom-metrics flag is now marked deprecated. (#54154, @mtaufen)

  • Specifying feature gates as a string of key-value pairs in ComponentConfig structures is awkward (#53024)

    • The Kubelet's feature gates are now specified as a map when provided via a JSON or YAML KubeletConfiguration, rather than as a string of key-value pairs. (#53025, @mtaufen)

NullArea

  • Local pods stay around after node deletion (#48213)

    • don't recreate a mirror pod for static pod when node gets deleted (#48339, @dixudx)

  • Extra CRI call during processing cpu set (#53304)

    • Only do UpdateContainerResources when cpuset is set (#53122, @resouer)

  • Cut and vendor cAdvisor v0.28.1 for the 1.9 release. (#55628)

  • Better handling of device plugin resource deletion (#53395)

    • Extends deviceplugin to gracefully handle full device plugin lifecycle. (#55088, @jiayingz)

  • Pods moving from Succeeded to Pending (#54499)

    • Fix an issue where pods were briefly transitioned to a "Pending" state during the deletion process. (#54593, @dashpole)

  • Remove the backward compatibility code for kubelet 1.2 in NodeController. (#48995)

    • The NodeController will not support kubelet 1.2. (#48996, @k82cn)

  • Deprecate --network-plugin-dir for kubelet (#46410)

  • FailedSync event from kubelet provides no value (#53900)

  • Should be able to specific unconfined AppArmor profile (#52370)

    • enable to specific unconfined AppArmor profile (#52395, @dixudx)

  • Consume ImageFS stats from StatsProvider in ImageGCManager (#53083)

    • Change ImageGCManage to consume ImageFS stats from StatsProvider (#53094, @yguo0905)

  • RFE: Bootstrap Checkpointing - Modify manifest behavior slightly for self hosting. (#49236)

  • Reduce cpumanager default logging verbosity (#54804)

    • set leveled logging (v=4) for 'updating container' message (#54865, @phsiao)

  • Hyperkube doesn't support --experimental-dockershim for kubelet (#54424)

  • CRI: Debug API (#53757)

    • Verbose option is added to each status function in CRI. Container runtime could return extra information in status response for debugging. (#53965, @Random-Liu)

  • Zone labels are removed every kubelet restart in 1.8 (#54070)

    • kubelet: prevent removal of default labels from Node API objects on startup (#54073, @liggitt)

  • NullIssue

    • Fix overlay2 container disk metrics for Docker and CRI-O (#54827, @dashpole)
    • BugFix: Exited containers are not Garbage Collected by the kubelet while the pod is running (#53167, @dashpole)
    • Add pod-level CPU and memory stats from pod cgroup information (#55969, @jingxu97)
    • Add pod-level local ephemeral storage metric in Summary API. Pod-level ephemeral storage reports the total filesystem usage for the containers and emptyDir volumes in the measured Pod. (#55447, @jingxu97)
    • Kubelet supports running mount utilities and final mount in a container instead running them on the host. (#53440, @jsafrane)
    • Remove docker dependency during kubelet start up (#54405, @resouer)
    • Add Windows support to the system verification check (#53730, @bsteciuk)
    • Don't remove extended resource capacities that are not registered with kubelet from node status. (#53353, @jiayingz)
    • RBAC: The default admin and edit roles now include read/write permissions and the view role includes read permissions on poddisruptionbudget.policy resources. (#52654, @liggitt)
    • Fix stats summary network value when multiple network interfaces are available. (#52144, @andyxning)
    • A new field is added to CRI container log format to support splitting a long log line into multiple lines. (#55922, @Random-Liu)
    • Kubelet can provide full summary api support except container log stats for CRI container runtime now. (#55810, @abhi)
    • Base images bumped to Debian Stretch (9) (#52744, @rphillips)
    • fluentd now supports CRI log format. (#54777, @Random-Liu)
    • Load kernel modules automatically inside a kube-proxy pod (#52003, @vfreex)
    • fix a bug where disk pressure could trigger prematurely when using overlay2 (#53684, @dashpole)
    • Metrics were added to network plugin to report latency of CNI operations (#53446, @sjenning)
    • Fix the bug that query Kubelet's stats summary with CRI stats enabled results in error. (#53107, @Random-Liu)

  • kubelet cannot show Docker-CE version correctly (#54039)

    • falls back to parse Docker runtime version as generic if not semver (#54040, @dixudx)

  • Large kubemark performance tests failing with timeout during ns deletion (#53327)

    • Optimize random string generator to avoid multiple locks & use bit-masking (#53720, @shyamjvs)
    • Use separate client for leader election in scheduler to avoid starving leader election by regular scheduler operations. (#53793, @wojtek-t)

  • [Failing Test] [k8s.io] Summary API when querying /stats/summary should report resource usage through the stats api (#55909)

  • [feature] for GPU and hugepages, default must match defaultRequest in LimitRange if both are specified (#54917)

    • validate if default and defaultRequest match when creating LimitRange for GPU and hugepages. (#54919, @tianshapjq)

  • Make CRI logs parsing to a library (#55136)

    • Add CRI log parsing library at pkg/kubelet/apis/cri/logs (#55140, @feiskyer)

  • Creation of gitRepo volume is broken in 1.8.0+ (#54129)

    • Add openssh-client back into the hyperkube image. This allows the gitRepo volume plugin to work properly. (#54250, @ixdy)

  • Error when using journald log driver and FallbackToLogsOnError (#52502)

    • Get fallback termination msg from docker when using journald log driver (#52503, @joelsmith)

Platform/Gce

  • Remove compute-rw scope from GCE nodes (#8074)

NullSig

Admission-Control

  • NullIssue

    • admission webhook registrations without a specific failure policy default to failing closed. (#54162, @deads2k)

Admission-Control & Security

  • NullIssue

    • The PodSecurityPolicy annotation kubernetes.io/psp on pods is only set once on create. (#55486, @sttts)

Audit

NullArea

  • NullIssue

    • The dynamic admission webhook now supports a URL in addition to a service reference, to accommodate out-of-cluster webhooks. (#54889, @lavalamp)
    • not calculate new priority when user update other spec of a pod (#55221, @CaoShuFeng)
    • The minimum supported go version bumps to 1.9.1. (#55301, @xiangpengzhao)
    • update podtolerations admission to mutate and validate separately (#55251, @deads2k)
    • Removes Priority Admission Controller from kubeadm since it's alpha. (#55237, @andrewsykim)
    • Add a new feature gate for enabling an alpha annotation which, if present, excludes the annotated node from being added to a service load balancers. (#54644, @brendandburns)
    • Add PodDisruptionBudget to scheduler cache. (#53914, @bsalamat)
    • Bugfix: master startup script on GCP no longer fails randomly due to concurrent iptables invocations. (#55945, @x13n)
    • Log when node is successfully initialized by Cloud Controller Manager (#53517, @andrewsykim)
    • kubeadm: Add support for adding a Windows node (#53553, @bsteciuk)
    • Added integration test for TaintNodeByCondition. (#53184, @k82cn)
    • If a non-absolute mountPath is passed to the kubelet, prefix it with the appropriate root path. (#55665, @brendandburns)
    • Added support for SAN entries in the master node certificate via juju kubernetes-master config. (#54234, @hyperbolic2346)
    • Add --etcd-compaction-interval to apiserver for controlling request of compaction to etcd3 from apiserver. (#51765, @mitake)
    • [cluster-monitoring addon] Update monitoring-influxdb-grafana to latest version (#53319, @kairen)
      • Improved generation of deb and rpm packages in bazel build (#53163, @kad)
    • Fix kubeadm upgrade plan for offline operation: ignore errors when trying to fetch latest versions from dl.k8s.io (#54016, @praseodym)
    • Update AWS SDK to 1.12.7 (#53561, @justinsb)
    • Optimize Repeated registration of AlgorithmProvider when ApplyFeatureGates (#54047, @kuramal)
    • Remove the LbaasV1 of OpenStack cloud provider, currently only support LbaasV2. (#52717, @FengyunPan)
    • Change kubeadm create token to default to the group that almost everyone will want to use. The group is system:bootstrappers:kubeadm:default-node-token and is the group that kubeadm sets up, via an RBAC binding, for auto-approval (system:certificates.k8s.io:certificatesigningrequests:nodeclient). (#53512, @jbeda)
    • Fixes a performance issue (#51899) identified in large-scale clusters when deleting thousands of pods simultaneously across hundreds of nodes, by actively removing containers of deleted pods, rather than waiting for periodic garbage collection and batching resulting pod API deletion requests. (#53233, @dashpole)
    • Avoid unnecessary spam in kube-controller-manager log if --cluster-cidr is not specified and --allocate-node-cidrs is false. (#54934, @akosiaris)
    • kubeadm: use the CRI for preflights checks (#55055, @runcom)
    • Fix clustered datastore name to be absolute. (#54438, @pshahzeb)
    • Add --no-negcache flag to kube-dns to prevent caching of NXDOMAIN responses. (#53604, @cblecker)
    • Kubelet evictions take pod priority into account (#53542, @dashpole)
    • Improve explanation of ReplicaSet (#53403, @rcorre)
    • default fail-swap-on to false for kubelet on kubernetes-worker charm (#53386, @wwwtyro)
    • Fix kubeadm reset crictl command (#55717, @runcom)
    • GCE: provide an option to disable docker's live-restore on COS/ubuntu (#55260, @yujuhong)
    • Addon manager supports HA masters. (#55466, @x13n)
    • Fix code-generators to produce correct code when GroupName, PackageName and/or GoName differ. (#55614, @sttts)
    • Metadata concealment on GCE is now controlled by the ENABLE_METADATA_CONCEALMENT env var. See cluster/gce/config-default.sh for more info. (#54150, @ihmccreery)
    • Update kube-dns 1.14.7 (#54443, @bowei)
    • Deprecation: The flag etcd-quorum-read of kube-apiserver is deprecated and the ability to switch off quorum read will be removed in a future release. (#53795, @xiangpengzhao)
    • Add extra-args configs for scheduler and controller-manager to kubernetes-master charm (#55185, @Cynerva)
    • The output of kubectl config get-contexts is now sorted alphabetically by the context name. (#46946, @kellycampbell)
    • Add extra-args configs to kubernetes-worker charm (#55334, @Cynerva)
    • Add masquerading rules by default to GCE/GKE (#55178, @dnardo)
    • Log error of failed healthz check (#53048, @mrIncompetent)
    • Update fluentd-gcp DaemonSet (#54175, @tallclair)
    • kubeadm: Strip bootstrap tokens from the kubeadm-config ConfigMap (#53559, @fabriziopandini)
    • Horizontal pod autoscaler uses REST clients through the kube-aggregator instead of the legacy client through the API server proxy. (#53205, @kawych)
    • Correct wording of kubeadm upgrade response for missing ConfigMap. (#53337, @jmhardison)
    • Fix metrics API group name in audit configuration (#53493, @piosz)
    • [fluentd-elasticsearch addon] Elasticsearch service name can be overridden via env variable ELASTICSEARCH_SERVICE_NAME (#54215, @mrahbar)
    • Allow HPA to read custom metrics. (#54854, @kawych)
    • In PodTolerationRestriction admisson plugin, if namespace level tolerations are empty, now they override cluster level tolerations. (#54812, @aveshagarwal)
    • secret data containing Docker registry auth objects is now generated using the config.json format (#53916, @juanvallejo)
    • Adding vishh as an reviewer/approver for hack directory (#54007, @vishh)
    • Use multi-arch busybox image for e2e (#54034, @dixudx)
    • Addon manager supports HA masters. (#55782, @x13n)
    • If you are using the cloud provider API to determine the external host address of the apiserver, set --external-hostname explicitly instead. The cloud provider detection has been deprecated and will be removed in the future (#54516, @dims)
    • Increase waiting time (120s) for docker startup in health-monitor.sh (#54099, @dchen1107)
    • kubeadm: Fix a bug on some OSes where the kubelet tried to mount a volume path that is non-existent and on a read-only filesystem (#55320, @andrewrynhard)
    • GCI mounter is moved from the manifests tarball to the server tarball. (#47497, @mikedanese)
    • kubeadm: reset: use crictl to reset containers (#54721, @runcom)
    • kubectl get will now use OpenAPI schema extensions by default to select columns for custom types. (#53483, @apelisse)
    • PodSecurityPolicies for addons (#55509, @tallclair)
    • Support completion for kubectl config rename-context (#48340, @superbrothers)
    • Add limitrange/resourcequota/downward_api e2e tests for local ephemeral storage (#52523, @NickrenREN)
    • Fix iptables FORWARD policy for Docker 1.13 in kubernetes-worker charm (#54796, @Cynerva)
    • Allow for configuring etcd hostname in the manifest (#54403, @wojtek-t)
    • Enable Priority admission control in kubeadm. (#53175, @andrewsykim)
    • Ignore extended resources that are not registered with kubelet during container resource allocation. (#53547, @jiayingz)
    • Upgrading the kubernetes-master units now results in staged upgrades just like the kubernetes-worker nodes. Use the upgrade action in order to continue the upgrade process on each unit such as juju run-action kubernetes-master/0 upgrade (#55990, @hyperbolic2346)
    • Added extra_sans config option to kubeapi-load-balancer charm. This allows the user to specify extra SAN entries on the certificate generated for the load balancer. (#54947, @hyperbolic2346)
    • Add support for RBAC support to Kubernetes via Juju (#53820, @ktsakalozos)
    • Support completion for --clusterrole of kubectl create clusterrolebinding (#48267, @superbrothers)
    • Fix permissions for Metrics Server. (#53330, @kawych)
    • Upgrade fluentd-elasticsearch addon to Elasticsearch/Kibana 5.6.2 (#53307, @aknuds1)
    • Added namespaceSelector to externalAdmissionWebhook configuration to allow applying webhooks only to objects in the namespaces that have matching labels. (#54727, @caesarxuchao)

Openstack

NullArea

  • OpenStack Cinder version detection fails (and is incorrect) (#50461)

  • NullIssue

Release

NullArea

Scalability

Etcd

NullArea

  • [PodSecurityPolicy] Optimize getMatchingPolicies (#55521)

    • Improved PodSecurityPolicy admission latency, but validation errors are no longer limited to only errors from authorized policies. (#55643, @tallclair)

  • NullIssue

    • ReplicationController now shares its underlying controller implementation with ReplicaSet to reduce the maintenance burden going forward. However, they are still separate resources and there should be no externally visible effects from this change. (#49429, @enisoc)
    • apiserver: --etcd-quorum-read now defaults to true, to ensure correct operation with HA etcd clusters (#53717, @liggitt)
    • API chunking via the limit and continue request parameters is promoted to beta in this release. Client libraries using the Informer or ListWatch types will automatically opt in to chunking. (#52949, @smarterclayton)

  • Update to Go 1.9 (#49484)

  • Large kubemark performance tests failing with timeout during ns deletion (#53327)

    • Use separate client for leader election in scheduler to avoid starving leader election by regular scheduler operations. (#53793, @wojtek-t)
    • Optimize random string generator to avoid multiple locks & use bit-masking (#53720, @shyamjvs)

Scheduling

Hw-Accelerators

  • NullIssue

    • Add ExtendedResourceToleration admission controller. This facilitates creation of dedicated nodes with extended resources. If operators want to create dedicated nodes with extended resources (like GPUs, FPGAs etc.), they are expected to taint the node with extended resource name as the key. This admission controller, if enabled, automatically adds tolerations for such taints to pods requesting extended resources, so users don't have to manually add these tolerations. (#55839, @mindprince)
    • GCE nodes with NVIDIA GPUs attached now expose nvidia.com/gpu as a resource instead of alpha.kubernetes.io/nvidia-gpu. (#54826, @mindprince)

NullArea

  • Consider moving TaintNodeUnreachable out of alpha (#54198)

  • Scheduler should handle pod updates during scheduling more gracefully (#52914)

    • Change scheduler to skip pod with updates only on pod annotations (#54008, @yguo0905)

  • Large kubemark performance tests failing with timeout during ns deletion (#53327)

    • Optimize random string generator to avoid multiple locks & use bit-masking (#53720, @shyamjvs)
    • Use separate client for leader election in scheduler to avoid starving leader election by regular scheduler operations. (#53793, @wojtek-t)

  • Support PodDisruptionBudget during preemption (#53913)

    • Add PodDisruptionBudget support during pod preemption (#56178, @bsalamat)

  • Fix starvation problem in pod preemption (#54501)

    • Add support to take nominated pods into account during scheduling to avoid starvation of higher priority pods. (#55933, @bsalamat)

  • Scheduler dies with "Schedulercache is corrupted" (#50916)

    • Fixed 'Schedulercache is corrupted' error in kube-scheduler (#55262, @liggitt)

  • Refactor kube-scheduler configuration (#52428)

    • The kube-scheduler command now supports a --config flag which is the location of a file containing a serialized scheduler configuration. Most other kube-scheduler flags are now deprecated. (#52562, @ironcladlou)

  • "notReady" toleration should be "not-ready" (#51246)

    • Use not-ready to replace notReady in node condition taint keys. (#51266, @resouer)

  • NullIssue

    • Add a new scheduling queue that helps schedule the highest priority pending pod first. (#55109, @bsalamat)
    • add hostIP and protocol to the original hostport predicates procedure in scheduler. (#52421, @WIZARD-CXY)
    • Object count quotas supported on all standard resources using count/<resource>.<group> syntax (#54320, @derekwaynecarr)
    • move getMaxVols function to predicates.go and add some NewVolumeCountPredicate funcs (#51783, @jiulongzaitian)
    • Apply algorithm in scheduler by feature gates. (#52723, @k82cn)
    • RBAC: The default admin and edit roles now include read/write permissions and the view role includes read permissions on poddisruptionbudget.policy resources. (#52654, @liggitt)
    • A new priority function ResourceLimitsPriorityMap (disabled by default and behind alpha feature gate and not part of the scheduler's default priority functions list) that assigns a lowest possible score of 1 to a node that satisfies one or both of input pod's cpu and memory limits, mainly to break ties between nodes with same scores. (#55906, @aveshagarwal)

  • Remove support for opaque integer resources (deprecated in v1.8) (#55102)

    • Remove opaque integer resources (OIR) support (deprecated in v1.8.) (#55103, @ConnorDoyle)

Service-Catalog

NullArea

  • PodPreset Feature Tests Consistently Causing ci-kubernetes-e2e-gci-gce-alpha-features to Fail (#53079)

    • Skip podpreset test if the alpha feature setttings/v1alpha1 is disabled (#53080, @jennybuckley)

Storage

NullArea

  • Taint a AWS node if a volume is stuck in "attaching" state for too long (#55502)

    • AWS: Apply taint to a node if volumes being attached to it are stuck in attaching state (#55558, @gnufied)

  • Remove ScaleIO dependency on drv_cfg binary for containerization (#54954)

    • ScaleIO driver completely removes dependency on drv_cfg binary so a Kubernetes cluster can easily run a containerized kubelet. (#54956, @vladimirvivien)

  • Detach is broken from stopped nodes in AWS (#55892)

  • should prevent the deletion of a PVC that is referenced by an active pod (#45143)

    • PVC Finalizing Controller is introduced in order to prevent deletion of a PVC that is being used by a pod. (#55824, @pospispa)

  • wrong controller-master detection (#54570)

  • Add support for verifying attached but desired to be detached PVCs (#52573)

    • Implement correction mechanism for dangling volumes attached for deleted pods (#55491, @gnufied)

  • Zero capacity PVs cause pods to fail and zero capacity PVCs create zero capacity PVs (#55553)

  • PersistentVolumeSource should be read-only (#54562)

    • Validate that PersistentVolumeSource is not changed during PV Update (#54761, @ianchakeres)

  • ScaleIO - credentials could be accessed by non-admin users (#53619)

    • ScaleIO persistent volumes now support referencing a secret in a namespace other than the bound persistent volume claim's namespace; this is controlled during provisioning with the secretNamespace storage class parameter; StoragePool and ProtectionDomain attributes no longer defaults to the value default (#54013, @vladimirvivien)

  • Recycle always failed on non x86 platform (#53942)

  • NullIssue

    • Pod Security Policy can now manage access to specific FlexVolume drivers (#53179, @wanghaoran1988)
    • Add resize support for ceph RBD (#52767, @NickrenREN)
    • Implement kubelet side file system resizing. Also implement GCE PD resizing (#55815, @gnufied)
    • Block volumes Support: CRI, volumemanager and operationexecutor changes (#51494, @mtanino)
    • RBD Persistent Volume Sources can now reference User's Secret in namespaces other than the namespace of the bound Persistent Volume Claim (#54302, @sbezverk)
    • Add support for resizing EBS disks (#56118, @gnufied)
    • Implement volume resize for cinder (#51498, @NickrenREN)
    • Block volumes Support: FC plugin update (#51493, @mtanino)
    • Updating vsphere cloud provider to support k8s cluster spread across multiple vCenters (#55845, @rohitjogvmw)
    • iSCSI Persistent Volume Sources can now reference CHAP Secrets in namespaces other than the namespace of the bound Persistent Volume Claim (#51530, @rootfs)
    • Kubelet supports running mount utilities and final mount in a container instead running them on the host. (#53440, @jsafrane)
    • allow windows mount path (#51240, @andyzhangx)

  • PVCs using standard StorageClass create PDs in disks in wrong zone in multi-zone GKE clusters (#50115)

    • Fix a bug in GCE multizonal clusters where PersistentVolumes were sometimes created in zones without nodes. (#52322, @davidz627)

  • Multi Attach PVC errors and events are too noisy (#53214)

Platform/Gce

  • Remove compute-rw scope from GCE nodes (#8074)

Testing

Federation

  • NullIssue

    • Development of Kubernetes Federation has moved to github.com/kubernetes/federation. This move out of tree also means that Federation will begin releasing separately from Kubernetes. The impact of this is Federation-specific behavior will no longer be included in kubectl, kubefed will no longer be released as part of Kubernetes, and the Federation servers will no longer be included in the hyperkube binary and image. (#53816, @marun)

Hw-Accelerators

  • NullIssue

    • Kubelet now exposes metrics for NVIDIA GPUs attached to the containers. (#55188, @mindprince)

NullArea

  • Update to Go 1.9 (#49484)

  • NullIssue

    • Fix to prevent downward api change break on older versions (#53673, @timothysc)
    • API chunking via the limit and continue request parameters is promoted to beta in this release. Client libraries using the Informer or ListWatch types will automatically opt in to chunking. (#52949, @smarterclayton)

Windows

NullArea

  • wrong controller-master detection (#54570)

  • there is azure file mount limit issue on windows due to using drive letter (#54668)

    • fix azure file mount limit issue on windows due to using drive letter (#53629, @andyzhangx)

  • there are lots of warning message due to GetMountRefs func in windows (#54670)

    • fix warning messages due to GetMountRefs func not implemented in windows (#52401, @andyzhangx)

  • NullIssue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment