Generated release notes for the entire 1.11 release

1.11.md

Action Required

• kubernetes-worker juju charm: Added support for setting the --enable-ssl-chain-completion option on the ingress proxy. "action required": if your installation relies on supplying incomplete certificate chains and using OCSP to fill them in, you must set "ingress-ssl-chain-completion" to "true" in your juju configuration. (#63845, @paulgear)
• A cluster-autoscaler ClusterRole is added to cover only the functionality required by Cluster Autoscaler and avoid abusing system:cluster-admin role. (#64503, @kgolab)
• [action required] TODO (#64792, @luxas)
• [action required] The structure of the kubelet dropin in the kubeadm deb package has changed significantly. (#64780, @luxas)
• [action required] The --node-name flag for kubeadm now dictates the Node API object name the (#64706, @liztio)
• [action required] kubeadm: The Token-related fields in the MasterConfiguration object have now been refactored. Instead of the top-level .Token, .TokenTTL, .TokenUsages, .TokenGroups fields, there is now a BootstrapTokens slice of BootstrapToken objects that support the same features under the .Token, .TTL, .Usages, .Groups fields. (#64408, @luxas)
• [action required] .NodeName and .CRISocket in the MasterConfiguration and NodeConfiguration v1alpha1 API objects are now .NodeRegistration.Name and .NodeRegistration.CRISocket respectively in the v1alpha2 API. The .NoTaintMaster field has been removed in the v1alpha2 API. (#64210, @luxas)
• (ACTION REQUIRED) PersisntVolumeLabel admission controller is now disabled by default. If you depend on this feature (AWS/GCE) then ensure it is added to the --enable-admission-plugins flag on the kube-apiserver. (#64326, @andrewsykim)
• [action required] kubeadm: The :Etcd struct has been refactored in the v1alpha2 API. All the options now reside under either .Etcd.Local or .Etcd.External. Automatic conversions from the v1alpha1 API are supported. (#64066, @luxas)
• [action required] kubeadm: kubelets in kubeadm clusters now disable the readonly port (10255). If you're relying on unauthenticated access to the readonly port, please switch to using the secure port (10250). Instead, you can now use ServiceAccount tokens when talking to the secure port, which will make it easier to get access to e.g. the /metrics endpoint of the kubelet securely. (#64187, @luxas)
• [action required] kubeadm: Support for .AuthorizationModes in the kubeadm v1alpha2 API has been removed. Instead, you can use the .APIServerExtraArgs and .APIServerExtraVolumes fields to achieve the same effect. Files using the v1alpha1 API and setting this field will be automatically upgraded to this v1alpha2 API and the information will be preserved. (#64068, @luxas)
• [action required] The formerly publicly-available cAdvisor web UI that the kubelet ran on port 4194 by default is now turned off by default. The flag configuring what port to run this UI on --cadvisor-port was deprecated in v1.10. Now the default is --cadvisor-port=0, in other words, to not run the web server. The recommended way to run cAdvisor if you still need it, is via a DaemonSet. The --cadvisor-port will be removed in v1.12 (#63881, @luxas)
• [action required] kubeadm: The .ImagePullPolicy field has been removed in the v1alpha2 API version. Instead it's set statically to IfNotPresent for all required images. If you want to always pull the latest images before cluster init (like what Always would do), run kubeadm config images pull before each kubeadm init. If you don't want the kubelet to pull any images at kubeadm init time, as you for instance don't have an internet connection, you can also run kubeadm config images pull before kubeadm init or side-load the images some other way (e.g. docker load -i image.tar). Having the images locally cached will result in no pull at runtime, which makes it possible to run without any internet connection. (#64096, @luxas)
• [action required] In the new v1alpha2 kubeadm Configuration API, the .CloudProvider and .PrivilegedPods fields don't exist anymore. (#63866, @luxas)
• [action required] kubeadm now uses an upgraded API version for the configuration file, kubeadm.k8s.io/v1alpha2. kubeadm in v1.11 will still be able to read v1alpha1 configuration, and will automatically convert the configuration to v1alpha2 internally and when storing the configuration in the ConfigMap in the cluster. (#63788, @luxas)
• The annotation service.alpha.kubernetes.io/tolerate-unready-endpoints is deprecated. Users should use Service.spec.publishNotReadyAddresses instead. (#63742, @thockin)
• ACTION REQUIRED: Alpha annotation for PersistentVolume node affinity has been removed. Update your PersistentVolumes to use the beta PersistentVolume.nodeAffinity field before upgrading to this release (#61816, @wackxu)

New Features

• Update Rescheduler's manifest to use version 0.4.0. (#65454, @bsalamat)
• Add user-agent to audit-logging. (#64812, @hzxuzhonghu)
• Add e2e regression tests for the kubelet being secure (#64140, @dixudx)
• Update crictl on GCE to v1.11.0. (#65254, @Random-Liu)
• All configured authorizers are now checked to determine if an RBAC role or clusterrole escalation (setting permissions the user does not currently have via RBAC) is allowed. (#56358, @liggitt)
• deprecated and inactive option '--enable-custom-metrics' is removed in 1.11 (#60699, @CaoShuFeng)
• kubeadm: Add a new kubeadm upgrade node config command (#64624, @luxas)
• Update version of Istio addon from 0.6.0 to 0.8.0. (#64537, @ostromart)
• Provides API support for external CSI storage drivers to support block volumes. (#64723, @vladimirvivien)
• [action required] TODO (#64792, @luxas)
• Introduce ContainersReady condition in Pod Status (#64646, @freehan)
• The Sysctls experimental feature has been promoted to beta (enabled by default via the Sysctls feature flag). PodSecurityPolicy and Pod objects now have fields for specifying and controlling sysctls. Alpha sysctl annotations will be ignored by 1.11+ kubelets. All alpha sysctl annotations in existing deployments must be converted to API fields to be effective. (#63717, @ingvagabund)
• [action required] The structure of the kubelet dropin in the kubeadm deb package has changed significantly. (#64780, @luxas)
• [action required] The --node-name flag for kubeadm now dictates the Node API object name the (#64706, @liztio)
• Volume topology aware dynamic provisioning (#63193, @lichuqiang)
• Updated Container Storage Interface specification version to v0.3.0 (#64719, @davidz627)
• Kubeadm: Make CoreDNS run in read-only mode and drop all unneeded privileges (#64473, @nberlee)
• Add a volume projection that is able to project service account tokens. (#62005, @mikedanese)
• PodSecurityPolicy now supports restricting hostPath volume mounts to be readOnly and under specific path prefixes (#58647, @jhorwit2)
• API change for volume topology aware dynamic provisioning (#63233, @lichuqiang)
• Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta (#63724, @ravisantoshgudimetla)
• Adding CSI driver registration code. (#64560, @sbezverk)
• client-go: credential exec plugins have been promoted to beta (#64482, @ericchiang)
• Revert #64364 to resurrect rescheduler. More info kubernetes/kubernetes#64725 :) (#64592, @ravisantoshgudimetla)
• Ability to quota resources by priority (#57963, @vikaschoudhary16)
• PodSecurityPolicy admission information is added to audit logs (#58143, @CaoShuFeng)
• Add Alpha support for dynamic volume limits based on node type (#64154, @gnufied)
• ScheduleDaemonSetPods is an alpha feature (since v1.11) that causes DaemonSet Pods (#63223, @k82cn)
• Kubelet now proxies container streaming between apiserver and container runtime. The connection between kubelet and apiserver is authenticated. Container runtime should change streaming server to serve on localhost, to make the connection between kubelet and container runtime local. (#64006, @Random-Liu)
• TokenRequests now are required to have an expiration duration between 10 minutes and 2^32 seconds. (#63999, @mikedanese)
• New conformance test added for Watch. (#61424, @jennybuckley)
• Implement kubelet side online file system resizing (#62460, @mlmhl)
• Remove rescheduler from master. (#64364, @ravisantoshgudimetla)
• Declare IPVS-based kube-proxy GA (#58442, @m1093782566)
• Exec authenticator plugin supports TLS client certificates. (#61803, @awly)
• Use Patch instead of Put to sync pod status (#62306, @freehan)
• Adds the VolumeSubpathEnvExpansion alpha feature to support environment variable expansion (#49388, @kevtaylor)
• kubeadm: Add a 'kubeadm config migrate' command to convert old API types to their newer counterparts in the new, supported API types. This is just a client-side tool, it just executes locally without requiring a cluster to be running. You can think about this as an Unix pipe that upgrades config files. (#64232, @luxas)
• [action required] .NodeName and .CRISocket in the MasterConfiguration and NodeConfiguration v1alpha1 API objects are now .NodeRegistration.Name and .NodeRegistration.CRISocket respectively in the v1alpha2 API. The .NoTaintMaster field has been removed in the v1alpha2 API. (#64210, @luxas)
• Add probe based mechanism for kubelet plugin discovery (#63328, @vikaschoudhary16)
• The dynamic Kubelet config feature is now beta, and the DynamicKubeletConfig feature gate is on by default. In order to use dynamic Kubelet config, ensure that the Kubelet's --dynamic-config-dir option is set. (#64275, @mtaufen)
• The Kubelet now exports metrics that report the assigned (node_config_assigned), last-known-good (node_config_last_known_good), and active (node_config_active) config sources, and a metric indicating whether the node is experiencing a config-related error (node_config_error). The config source metrics always report the value 1, and carry the node_config_name, node_config_uid, node_config_resource_version, and node_config_kubelet_key labels, which identify the config version. The error metric reports 1 if there is an error, 0 otherwise. (#57527, @mtaufen)
• Add CRD Versioning with NOP converter (#63830, @mbohlool)
• The dynamic Kubelet config feature will now update config in the event of a ConfigMap mutation, which reduces the chance for silent config skew. Only name, namespace, and kubeletConfigKey may now be set in Node.Spec.ConfigSource.ConfigMap. The least disruptive pattern for config management is still to create a new ConfigMap and incrementally roll out a new Node.Spec.ConfigSource. (#63221, @mtaufen)
• Kubernetes cluster on GCE have crictl installed now. Users can use it to help debug their node. The documentation of crictl can be found https://github.com/kubernetes-incubator/cri-tools/blob/master/docs/crictl.md. (#63357, @Random-Liu)
• The status of dynamic Kubelet config is now reported via Node.Status.Config, rather than the KubeletConfigOk node condition. (#63314, @mtaufen)
• The PriorityClass API is promoted to scheduling.k8s.io/v1beta1 (#63100, @ravisantoshgudimetla)
• You must now specify Node.Spec.ConfigSource.ConfigMap.KubeletConfigKey when using dynamic Kubelet config to tell the Kubelet which key of the ConfigMap identifies its config file. (#59847, @mtaufen)
• StorageObjectInUseProtection feature is GA. (#62870, @pospispa)
• Split PodPriority and PodPreemption feature gate (#62243, @resouer)
• Add write-config-to to scheduler (#62515, @resouer)
• Add prometheus cluster monitoring addon to kube-up (#62195, @serathius)
• cluster/kube-up.sh now provisions a Kubelet config file for GCE via the metadata server. This file is installed by the corresponding GCE init scripts. (#62183, @mtaufen)
• Upgrade the default etcd server version to 3.2.18 (#61198, @jpbetz)
• kube-apiserver: oidc authentication now supports requiring specific claims with --oidc-required-claim=<claim>=<value> (#62136, @rithujohn191)
• Schedule even if extender is not available when using extender (#61445, @resouer)
• OIDC authentication now allows tokens without an "email_verified" claim when using the "email" claim. If an "email_verified" claim is present when using the "email" claim, it must be true. (#61508, @rithujohn191)
• add kubectl api-resources command to discovery of resources (#42873, @xilabao)
• Update to use go1.10.1 (#60597, @cblecker)
• The node authorizer now automatically sets up rules for Node.Spec.ConfigSource when the DynamicKubeletConfig feature gate is enabled. (#60100, @mtaufen)

API Changes

• Add user-agent to audit-logging. (#64812, @hzxuzhonghu)
• The deprecated --service-account-private-key-file flag has been removed from the cloud-controller-manager. The flag is still present and supported in the kube-controller-manager. (#65182, @liggitt)
• Introduce ContainersReady condition in Pod Status (#64646, @freehan)
• The Sysctls experimental feature has been promoted to beta (enabled by default via the Sysctls feature flag). PodSecurityPolicy and Pod objects now have fields for specifying and controlling sysctls. Alpha sysctl annotations will be ignored by 1.11+ kubelets. All alpha sysctl annotations in existing deployments must be converted to API fields to be effective. (#63717, @ingvagabund)
• PodSecurityPolicy now supports restricting hostPath volume mounts to be readOnly and under specific path prefixes (#58647, @jhorwit2)
• API change for volume topology aware dynamic provisioning (#63233, @lichuqiang)
• Ability to quota resources by priority (#57963, @vikaschoudhary16)
• Add Alpha support for dynamic volume limits based on node type (#64154, @gnufied)
• Add spec. additionalPrinterColumns to CRDs to define server side printing columns. (#60991, @sttts)
• Use DeleteOptions.PropagationPolicy instead of OrphanDependents in kubectl (#59851, @nilebox)
• The status of dynamic Kubelet config is now reported via Node.Status.Config, rather than the KubeletConfigOk node condition. (#63314, @mtaufen)
• The annotation service.alpha.kubernetes.io/tolerate-unready-endpoints is deprecated. Users should use Service.spec.publishNotReadyAddresses instead. (#63742, @thockin)
• You must now specify Node.Spec.ConfigSource.ConfigMap.KubeletConfigKey when using dynamic Kubelet config to tell the Kubelet which key of the ConfigMap identifies its config file. (#59847, @mtaufen)
• kube-apiserver: --endpoint-reconciler-type now defaults to lease. The master-count reconciler is deprecated and will be removed in 1.13. (#58474, @rphillips)
• Added MatchFields to NodeSelectorTerm; in 1.11, it only support metadata.name. (#62002, @k82cn)
• Split PodPriority and PodPreemption feature gate (#62243, @resouer)
• Add support to resize Portworx volumes. (#62308, @harsh-px)
• Kubelets will no longer set externalID in their node spec. (#61877, @mikedanese)
• Pod affinity nodeSelectorTerm.matchExpressions may now be empty, and works as previously documented: nil or empty matchExpressions matches no objects in scheduler. (#62448, @k82cn)
• Default mount propagation has changed from "HostToContainer" ("rslave" in Linux terminology) to "None" ("private") to match the behavior in 1.9 and earlier releases. "HostToContainer" as a default caused regressions in some pods. (#62462, @jsafrane)
• Remove ObjectMeta ListOptions DeleteOptions from core api group. Please use that in meta/v1 (#61809, @hzxuzhonghu)
• Support for "alpha.kubernetes.io/nvidia-gpu" resource which was deprecated in 1.10 is removed. Please use the resource exposed by DevicePlugins instead ("nvidia.com/gpu"). (#61498, @mindprince)
• ACTION REQUIRED: Alpha annotation for PersistentVolume node affinity has been removed. Update your PersistentVolumes to use the beta PersistentVolume.nodeAffinity field before upgrading to this release (#61816, @wackxu)
• kube-scheduler has been fixed to use --leader-elect option back to true (as it was in previous versions) (#59732, @dims)
• Display requests/limits of extended resources in node allocated resources. (#46079, @xiangpengzhao)
• Automatically add system critical priority classes at cluster boostrapping. (#60519, @bsalamat)
• fix sorting taints in case the sorting keys are equal (#61255, @dixudx)
• NetworkPolicies can now target specific pods in other namespaces by including both a namespaceSelector and a podSelector in the same peer element. (#60452, @danwinship)

SIG API Machinery

• Etcd health checks by the apiserver now ensure the apiserver can connect to and exercise the etcd API (#65027, @liggitt)
• fixed incorrect OpenAPI schema for CustomResourceDefinition objects (#65256, @liggitt)
• The deprecated --service-account-private-key-file flag has been removed from the cloud-controller-manager. The flag is still present and supported in the kube-controller-manager. (#65182, @liggitt)
• Webhooks for the mutating admission controller now support "remove" operation. (#64255, @rojkov)
• Orphan delete is now supported for custom resources (#63386, @roycaihw)
• fixes a potential deadlock in the garbage collection controller (#64235, @liggitt)
• Meta data of CustomResources is now pruned and schema checked during deserialization of requests and when read from etcd. In the former case, invalid meta data is rejected, in the later it is dropped from the CustomResource objects. (#64267, @sttts)
• Expose /debug/flags/v to allow dynamically set glog logging level, if want to change glog level to 3, you only have to send a PUT request with like curl -X PUT http://127.0.0.1:8080/debug/flags/v -d "3". (#63777, @hzxuzhonghu)
• New conformance test added for Watch. (#61424, @jennybuckley)
• Exec authenticator plugin supports TLS client certificates. (#61803, @awly)
• Add Establishing Controller on CRDs to avoid race between Established condition and CRs actually served. In HA setups, the Established condition is delayed by 5 seconds. (#63068, @xmudrii)
• Add spec. additionalPrinterColumns to CRDs to define server side printing columns. (#60991, @sttts)
• API server properly parses propagationPolicy as a query parameter sent with a delete request (#63414, @roycaihw)
• Property serverAddressByClientCIDRs in metav1.APIGroup (discovery API) now become optional instead of required (#61963, @roycaihw)
• The kube-apiserver openapi doc now includes extensions identifying APIService and CustomResourceDefinition kinds (#64174, @liggitt)
• apiservices/status and certificatesigningrequests/status now support GET and PATCH (#64063, @roycaihw)
• Add CRD Versioning with NOP converter (#63830, @mbohlool)
• kubectl and client-go now detects duplicated name for user, cluster and context when loading kubeconfig and reports error (#60464, @roycaihw)
• Subresources for custom resources is now beta and enabled by default. With this, updates to the /status subresource will disallow updates to all fields other than .status (not just .spec and .metadata as before). Also, required can be used at the root of the CRD OpenAPI validation schema when the /status subresource is enabled. (#63598, @nikhita)
• HTTP transport now uses context.Context to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a context.Context parameter. This is a breaking change if you use this field in your code. (#60012, @ash2k)
• list/watch API requests with a fieldSelector that specifies metadata.name can now be authorized as requests for an individual named resource (#63469, @wojtek-t)
• kubelet: fix hangs in updating Node status after network interruptions/changes between the kubelet and API server (#63492, @liggitt)
• The old dynamic client has been replaced by a new one. The previous dynamic client will exist for one release in client-go/deprecated-dynamic. Switch as soon as possible. (#63446, @deads2k)
• CustomResourceDefinitions Status subresource now supports GET and PATCH (#63619, @roycaihw)
• GC is now bound by QPS (it wasn't before) and so if you need more QPS to avoid ratelimiting GC, you'll have to set it. (#63657, @shyamjvs)
• the shortcuts which were moved server-side in at least 1.9 have been removed from being hardcoded in kubectl (#63507, @deads2k)
• owner references can be set during creation without deletion power (#63403, @deads2k)
• kube-apiserver: --endpoint-reconciler-type now defaults to lease. The master-count reconciler is deprecated and will be removed in 1.13. (#58474, @rphillips)
• KUBE_API_VERSIONS is no longer respected. It was used for testing, but runtime-config is the proper flag to set. (#63165, @deads2k)
• client-go developers: the new dynamic client is easier to use and the old is deprecated, you must switch. (#62913, @deads2k)
• Code generated for CRDs now passes go vet. (#62412, @bhcleek)
• Upgrade the default etcd server version to 3.2.18 (#61198, @jpbetz)
• Add e2e test for CRD Watch (#61025, @ayushpateria)

SIG Apps

• ScheduleDaemonSetPods is an alpha feature (since v1.11) that causes DaemonSet Pods (#63223, @k82cn)
• kubectl: This client version requires the apps/v1 APIs, so it will not work against a cluster version older than v1.9.0. Note that kubectl only guarantees compatibility with clusters that are +/-1 minor version away. (#61419, @enisoc)
• event object references with apiversion will now report an apiversion. (#63913, @deads2k)

SIG Architecture

• Add e2e regression tests for the kubelet being secure (#64140, @dixudx)
• New conformance test added for Watch. (#61424, @jennybuckley)

SIG Auth

• Add user-agent to audit-logging. (#64812, @hzxuzhonghu)
• Add e2e regression tests for the kubelet being secure (#64140, @dixudx)
• All configured authorizers are now checked to determine if an RBAC role or clusterrole escalation (setting permissions the user does not currently have via RBAC) is allowed. (#56358, @liggitt)
• The Sysctls experimental feature has been promoted to beta (enabled by default via the Sysctls feature flag). PodSecurityPolicy and Pod objects now have fields for specifying and controlling sysctls. Alpha sysctl annotations will be ignored by 1.11+ kubelets. All alpha sysctl annotations in existing deployments must be converted to API fields to be effective. (#63717, @ingvagabund)
• Add a volume projection that is able to project service account tokens. (#62005, @mikedanese)
• Fix kubectl auth can-i exit code. It will return 1 if the user is not allowed and 0 if it's allowed. (#59579, @fbac)
• PodSecurityPolicy now supports restricting hostPath volume mounts to be readOnly and under specific path prefixes (#58647, @jhorwit2)
• client-go: credential exec plugins have been promoted to beta (#64482, @ericchiang)
• PodSecurityPolicy admission information is added to audit logs (#58143, @CaoShuFeng)
• Kubelet now proxies container streaming between apiserver and container runtime. The connection between kubelet and apiserver is authenticated. Container runtime should change streaming server to serve on localhost, to make the connection between kubelet and container runtime local. (#64006, @Random-Liu)
• TokenRequests now are required to have an expiration duration between 10 minutes and 2^32 seconds. (#63999, @mikedanese)
• Exec authenticator plugin supports TLS client certificates. (#61803, @awly)
• [action required] kubeadm: kubelets in kubeadm clusters now disable the readonly port (10255). If you're relying on unauthenticated access to the readonly port, please switch to using the secure port (10250). Instead, you can now use ServiceAccount tokens when talking to the secure port, which will make it easier to get access to e.g. the /metrics endpoint of the kubelet securely. (#64187, @luxas)
• kubectl auth reconcile only works with rbac.v1 (#63967, @deads2k)
• [action required] The formerly publicly-available cAdvisor web UI that the kubelet ran on port 4194 by default is now turned off by default. The flag configuring what port to run this UI on --cadvisor-port was deprecated in v1.10. Now the default is --cadvisor-port=0, in other words, to not run the web server. The recommended way to run cAdvisor if you still need it, is via a DaemonSet. The --cadvisor-port will be removed in v1.12 (#63881, @luxas)
• list/watch API requests with a fieldSelector that specifies metadata.name can now be authorized as requests for an individual named resource (#63469, @wojtek-t)
• The NodeRestriction admission plugin now prevents kubelets from modifying/removing taints applied to their Node API object. (#63167, @liggitt)
• corrects a race condition in bootstrapping aggregated cluster roles in new HA clusters (#63761, @liggitt)
• Lays groundwork for OIDC distributed claims handling in the apiserver authentication token checker. (#63213, @filmil)
• kubelets are no longer allowed to delete their own Node API object. Prior to 1.11, in rare circumstances related to cloudprovider node ID changes, kubelets would attempt to delete/recreate their Node object at startup. If a legacy kubelet encounters this situation, a cluster admin can remove the Node object: (#62818, @mikedanese)
• Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events. (#61711, @crassirostris)
• Remove alpha functionality that allowed the controller manager to approve kubelet server certificates. (#62471, @mikedanese)
• Resolves forbidden error when the daemon-set-controller cluster role access controllerrevisions resources. (#62146, @frodenas)
• kube-apiserver: oidc authentication now supports requiring specific claims with --oidc-required-claim=<claim>=<value> (#62136, @rithujohn191)
• Add warnings that authors of aggregated API servers must not rely on authorization being done by the kube-apiserver. (#61349, @sttts)
• OIDC authentication now allows tokens without an "email_verified" claim when using the "email" claim. If an "email_verified" claim is present when using the "email" claim, it must be true. (#61508, @rithujohn191)

SIG Autoscaling

• Updates Cluster Autoscaler version to 1.3.0. Release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.3.0 (#65219, @aleksandra-malinowska)
• Update Cluster Autoscaler to v1.3.0-beta.2. Release notes for this version: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.3.0-beta.2 (#65148, @aleksandra-malinowska)
• A cluster-autoscaler ClusterRole is added to cover only the functionality required by Cluster Autoscaler and avoid abusing system:cluster-admin role. (#64503, @kgolab)
• Update Cluster Autoscaler version to 1.3.0-beta.1. Release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.3.0-beta.1 (#64977, @aleksandra-malinowska)
• Cluster Autoscaler 1.2.2 (release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.2.2) (#63974, @aleksandra-malinowska)
• Cluster Autoscaler 1.2.1 (release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.2.1) (#62457, @mwielgus)
• Cluster Autoscaler 1.2.0 - release notes available here: https://github.com/kubernetes/autoscaler/releases (#61561, @mwielgus)

SIG AWS

• AWS now checks for validity of ecryption key when creating encrypted volumes. Dynamic provisioning of encrypted volume may get slower due to these checks. (#65223, @jsafrane)
• AWS EBS volumes can be now used as ReadOnly in pods. (#64403, @jsafrane)
• (ACTION REQUIRED) PersisntVolumeLabel admission controller is now disabled by default. If you depend on this feature (AWS/GCE) then ensure it is added to the --enable-admission-plugins flag on the kube-apiserver. (#64326, @andrewsykim)

SIG Azure

• Fix kube-controller-manager panic while provisioning Azure security group rules (#64739, @feiskyer)
• Add reason message logs for non-exist Azure resources (#64248, @feiskyer)
• Fix SessionAffinity not updated issue for Azure load balancer (#64180, @feiskyer)
• Do not check vmSetName when getting Azure node's IP (#63541, @feiskyer)
• Azure VMSS: support VM names to contain the _ character (#63526, @djsly)
• Upgrade Azure Go SDK to stable version (v14.6.0) (#63063, @feiskyer)
• Add support of standard LB to Azure vmss (#62707, @feiskyer)
• Ensure expected load balancer is selected for Azure (#62450, @feiskyer)
• Azure cloud provider now supports standard SKU load balancer and public IP. To use it, set cloud provider config with (#61884, @feiskyer)
• Ensure cloudprovider.InstanceNotFound is reported when the VM is not found on Azure (#61531, @feiskyer)
• Azure cloud provider now supports specifying allowed service tags by annotation service.beta.kubernetes.io/azure-allowed-service-tags (#61467, @feiskyer)

SIG CLI

• Using kubectl describe on CRDs that use underscores will be prettier. (#65391, @smarterclayton)
• Fixes missing nodes lines when kubectl top nodes (#64389, @yue9944882)
• Fix kubectl drain --timeout option when eviction is used. (#64378, @wrdls)
• kubectl will list all allowed print formats when an invalid format is passed. (#64371, @CaoShuFeng)
• Fix kubectl auth can-i exit code. It will return 1 if the user is not allowed and 0 if it's allowed. (#59579, @fbac)
• apply global flag "context" for kubectl config view --minify (#64608, @dixudx)
• The --dry-run flag has been enabled for kubectl auth reconcile (#64458, @mrogers950)
• Use DeleteOptions.PropagationPolicy instead of OrphanDependents in kubectl (#59851, @nilebox)
• kubectl create [secret | configmap] --from-file now works on Windows with fully-qualified paths (#63439, @liggitt)
• kubectl api-resources now supports filtering to resources supporting specific verbs, and can output fully qualified resource names suitable for combining with commands like kubectl get (#63254, @liggitt)
• Deprecate kubectl rolling-update (#61285, @soltysh)
• kubectl stops rendering List as suffix kind name for CRD resources (#62512, @dixudx)
• Implements verbosity logging feature for kubeadm commands (#57661, @vbmade2000)
• Fixed column alignment when kubectl get is used with custom columns from OpenAPI schema (#56629, @luksa)
• add kubectl api-resources command to discovery of resources (#42873, @xilabao)
• kubectl: improves compatibility with older servers when creating/updating API objects (#61949, @liggitt)
• kubectl: restore the ability to show resource kinds when displaying multiple objects (#61985, @liggitt)
• Ensure reasons end up as comments in kubectl edit. (#60990, @bmcstdio)
• kubectl: fixes issue with -o yaml and -o json omitting kind and apiVersion when used with --dry-run (#61808, @liggitt)
• Display requests/limits of extended resources in node allocated resources. (#46079, @xiangpengzhao)
• escape literal percent sign when formatting (#61523, @dixudx)
• kubectl patch now supports --dry-run. (#60675, @timoreimann)
• fix sorting taints in case the sorting keys are equal (#61255, @dixudx)
• include node internal ip as additional information for kubectl (#57623, @dixudx)

SIG Cluster Lifecycle

• kubeadm: notify the user of manifest upgrade timeouts (#65164, @xlgao-zju)
• kubeadm: Use the release-1.11 branch by default (#65229, @luxas)
• kubeadm - Ensure the peer port is secured by explicitly setting the peer URLs for the default etcd instance. (#64988, @detiber)
• kubernetes now packages cri-tools (crictl) in addition to all the other kubeadm tools in a deb and rpm. (#64836, @chuckha)
• kubeadm: only run kube-proxy on architecture consistent nodes (#64696, @dixudx)
• kubeadm: Add a new kubeadm upgrade node config command (#64624, @luxas)
• Update version of Istio addon from 0.6.0 to 0.8.0. (#64537, @ostromart)
• [action required] TODO (#64792, @luxas)
• kubeadm now configures the etcd liveness probe correctly when etcd is listening on all interfaces (#64670, @stealthybox)
• [action required] The structure of the kubelet dropin in the kubeadm deb package has changed significantly. (#64780, @luxas)
• [action required] The --node-name flag for kubeadm now dictates the Node API object name the (#64706, @liztio)
• CoreDNS deployment configuration now uses k8s.gcr.io imageRepository (#64775, @rajansandeep)
• Kubeadm: Make CoreDNS run in read-only mode and drop all unneeded privileges (#64473, @nberlee)
• Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta (#63724, @ravisantoshgudimetla)
• [action required] kubeadm: The Token-related fields in the MasterConfiguration object have now been refactored. Instead of the top-level .Token, .TokenTTL, .TokenUsages, .TokenGroups fields, there is now a BootstrapTokens slice of BootstrapToken objects that support the same features under the .Token, .TTL, .Usages, .Groups fields. (#64408, @luxas)
• kubeadm upgrade apply can now ignore version errors with --force (#64570, @liztio)
• Kubelet now proxies container streaming between apiserver and container runtime. The connection between kubelet and apiserver is authenticated. Container runtime should change streaming server to serve on localhost, to make the connection between kubelet and container runtime local. (#64006, @Random-Liu)
• kubeadm now preserves previous manifests after upgrades (#64337, @liztio)
• kubeadm: conditionally set the kubelet cgroup driver for Docker (#64347, @neolit123)
• Label ExternalEtcdClientCertificates can be used for ignoring all preflight check issues related to client certificate files for external etcd. (#64269, @kad)
• kubeadm: Add a 'kubeadm config migrate' command to convert old API types to their newer counterparts in the new, supported API types. This is just a client-side tool, it just executes locally without requiring a cluster to be running. You can think about this as an Unix pipe that upgrades config files. (#64232, @luxas)
• [action required] .NodeName and .CRISocket in the MasterConfiguration and NodeConfiguration v1alpha1 API objects are now .NodeRegistration.Name and .NodeRegistration.CRISocket respectively in the v1alpha2 API. The .NoTaintMaster field has been removed in the v1alpha2 API. (#64210, @luxas)
• CoreDNS is now v1.1.3 (#64258, @rajansandeep)
• kubeadm will pull required images during preflight checks if it cannot find them on the system (#64105, @chuckha)
• kubeadm: rename the addon parameter kube-dns to coredns for kubeadm alpha phases addons as CoreDNS is now the default DNS server in 1.11. (#64274, @neolit123)
• kubeadm: when starting the API server use the arguments --enable-admission-plugins and --disable-admission-plugins instead of the deprecated --admission-control. (#64165, @neolit123)
• [action required] kubeadm: The :Etcd struct has been refactored in the v1alpha2 API. All the options now reside under either .Etcd.Local or .Etcd.External. Automatic conversions from the v1alpha1 API are supported. (#64066, @luxas)
• [action required] kubeadm: kubelets in kubeadm clusters now disable the readonly port (10255). If you're relying on unauthenticated access to the readonly port, please switch to using the secure port (10250). Instead, you can now use ServiceAccount tokens when talking to the secure port, which will make it easier to get access to e.g. the /metrics endpoint of the kubelet securely. (#64187, @luxas)
• Correct the way we reset containers and pods in kubeadm via crictl (#63862, @runcom)
• The kubelet certificate rotation feature can now be enabled via the .RotateCertificates field in the kubelet's config file. The --rotate-certificates flag is now deprecated, and will be removed in a future release. (#63912, @luxas)
• "kubeadm init" now writes a structured and versioned kubelet ComponentConfiguration file to /var/lib/kubelet/config.yaml and an environment file with runtime flags (you can source this file in the systemd kubelet dropin) to /var/lib/kubelet/kubeadm-flags.env. (#63887, @luxas)
• [action required] kubeadm: Support for .AuthorizationModes in the kubeadm v1alpha2 API has been removed. Instead, you can use the .APIServerExtraArgs and .APIServerExtraVolumes fields to achieve the same effect. Files using the v1alpha1 API and setting this field will be automatically upgraded to this v1alpha2 API and the information will be preserved. (#64068, @luxas)
• The dynamic Kubelet config feature will now update config in the event of a ConfigMap mutation, which reduces the chance for silent config skew. Only name, namespace, and kubeletConfigKey may now be set in Node.Spec.ConfigSource.ConfigMap. The least disruptive pattern for config management is still to create a new ConfigMap and incrementally roll out a new Node.Spec.ConfigSource. (#63221, @mtaufen)
• [action required] The formerly publicly-available cAdvisor web UI that the kubelet ran on port 4194 by default is now turned off by default. The flag configuring what port to run this UI on --cadvisor-port was deprecated in v1.10. Now the default is --cadvisor-port=0, in other words, to not run the web server. The recommended way to run cAdvisor if you still need it, is via a DaemonSet. The --cadvisor-port will be removed in v1.12 (#63881, @luxas)
• [action required] kubeadm: The .ImagePullPolicy field has been removed in the v1alpha2 API version. Instead it's set statically to IfNotPresent for all required images. If you want to always pull the latest images before cluster init (like what Always would do), run kubeadm config images pull before each kubeadm init. If you don't want the kubelet to pull any images at kubeadm init time, as you for instance don't have an internet connection, you can also run kubeadm config images pull before kubeadm init or side-load the images some other way (e.g. docker load -i image.tar). Having the images locally cached will result in no pull at runtime, which makes it possible to run without any internet connection. (#64096, @luxas)
• kubeadm: A kubeadm config print-default command has now been added that you can use as a starting point when writing your own kubeadm configuration files (#63969, @luxas)
• Update kubeadm's minimum supported kubernetes in v1.11.x to 1.10 (#63920, @dixudx)
• kubeadm has removed .Etcd.SelfHosting from its configuration API. It was never used in practice. (#63871, @luxas)
• adds the kubeadm upgrade diff command to show how static pod manifests will be changed by an upgrade. (#63930, @liztio)
• [action required] In the new v1alpha2 kubeadm Configuration API, the .CloudProvider and .PrivilegedPods fields don't exist anymore. (#63866, @luxas)
• Adds a kubeadm config images pull command to pull container images used by kubeadm. (#63833, @chuckha)
• [action required] kubeadm now uses an upgraded API version for the configuration file, kubeadm.k8s.io/v1alpha2. kubeadm in v1.11 will still be able to read v1alpha1 configuration, and will automatically convert the configuration to v1alpha2 internally and when storing the configuration in the ConfigMap in the cluster. (#63788, @luxas)
• Kubernetes cluster on GCE have crictl installed now. Users can use it to help debug their node. The documentation of crictl can be found https://github.com/kubernetes-incubator/cri-tools/blob/master/docs/crictl.md. (#63357, @Random-Liu)
• The NodeRestriction admission plugin now prevents kubelets from modifying/removing taints applied to their Node API object. (#63167, @liggitt)
• The status of dynamic Kubelet config is now reported via Node.Status.Config, rather than the KubeletConfigOk node condition. (#63314, @mtaufen)
• kubeadm now checks that IPv4/IPv6 forwarding is enabled (#63872, @kad)
• kubeadm will now deploy CoreDNS by default instead of KubeDNS (#63509, @detiber)
• kubeadm upgrade now supports external etcd setups again (#63495, @detiber)
• kubeadm will no longer generate an unused etcd CA and certificates when configured to use an external etcd cluster. (#63806, @detiber)
• The old dynamic client has been replaced by a new one. The previous dynamic client will exist for one release in client-go/deprecated-dynamic. Switch as soon as possible. (#63446, @deads2k)
• Removes a preflight check for kubeadm that validated custom kube-apiserver, kube-controller-manager and kube-scheduler arguments. (#63673, @chuckha)
• Adds a list-images subcommand to kubeadm that lists required images for a kubeadm install. (#63450, @chuckha)
• You must now specify Node.Spec.ConfigSource.ConfigMap.KubeletConfigKey when using dynamic Kubelet config to tell the Kubelet which key of the ConfigMap identifies its config file. (#59847, @mtaufen)
• Fixed where we get latest builds for stable branches (#63504, @dims)
• Search standard KubeConfig file locations when using kubeadm token without --kubeconfig. (#62850, @neolit123)
• kubeadm upgrade plan now accepts a version which improves the UX nicer in air-gapped environments. (#63201, @chuckha)
• kube-apiserver: --endpoint-reconciler-type now defaults to lease. The master-count reconciler is deprecated and will be removed in 1.13. (#58474, @rphillips)
• kubeadm: prompt the user for confirmation when resetting a master node (#59115, @alexbrand)
• kubeadm creates kube-proxy with a toleration to run on all nodes, no matter the taint. (#62390, @discordianfish)
• Mount additional paths required for a working CA root, for setups where /etc/ssl/certs doesn't contains certificates but just symlink. (#59122, @klausenbusk)
• kubeadm upgrade no longer races leading to unexpected upgrade behavior on pod restarts (#62655, @stealthybox)
• Kubelets will no longer set externalID in their node spec. (#61877, @mikedanese)
• kubeadm preflight: check CRI socket path if defined, otherwise check for Docker (#62481, @taharah)
• fixes configuration error when upgrading kubeadm from 1.9 to 1.10+ (#62568, @liztio)
• Phase kubeadm alpha phase kubelet is added to support dynamic kubelet configuration in kubeadm. (#57224, @xiangpengzhao)
• kubeadm alpha phase kubeconfig user supports groups (organizations) to be specified in client cert. (#62627, @xiangpengzhao)
• The kubeadm config option API.ControlPlaneEndpoint has been extended to take an optional port which may differ from the apiserver's bind port. (#62314, @rjosephwright)
• cluster/kube-up.sh now provisions a Kubelet config file for GCE via the metadata server. This file is installed by the corresponding GCE init scripts. (#62183, @mtaufen)
• Adds --cluster-name to kubeadm init for specifying the cluster name in kubeconfig. (#60852, @karan)
• Upgrade the default etcd server version to 3.2.18 (#61198, @jpbetz)
• Fixed #731 kubeadm upgrade ignores HighAvailability feature gate (#62455, @fabriziopandini)
• kubeadm: surface external etcd preflight validation errors (#60585, @alexbrand)
• Implements verbosity logging feature for kubeadm commands (#57661, @vbmade2000)
• kubeadm: Introduce join timeout that can be controlled via the discoveryTimeout config option (set to 5 minutes by default). (#60983, @rosti)

SIG GCP

• User can now use sudo crictl on GCE cluster. (#65389, @Random-Liu)
• Update crictl on GCE to v1.11.0. (#65254, @Random-Liu)
• GCE: Fixes operation polling to adhere to the specified interval. Furthermore, operation errors are now returned instead of ignored. (#64630, @nicksardo)
• Update version of Istio addon from 0.6.0 to 0.8.0. (#64537, @ostromart)
• GCE: Update cloud provider to use TPU v1 API (#64727, @yguo0905)
• (ACTION REQUIRED) PersisntVolumeLabel admission controller is now disabled by default. If you depend on this feature (AWS/GCE) then ensure it is added to the --enable-admission-plugins flag on the kube-apiserver. (#64326, @andrewsykim)
• Upgrade the default etcd server version to 3.2.18 (#61198, @jpbetz)

SIG Instrumentation

• Stop using InfluxDB as default cluster monitoring (#62328, @serathius)
• Add prometheus cluster monitoring addon to kube-up (#62195, @serathius)
• [fluentd-gcp addon] Partial success option is enabled in fluentd. (#61773, @crassirostris)
• [fluentd-gcp addon] Update event-exporter image to have the latest base image. (#61727, @crassirostris)

SIG Network

• Allow access to ClusterIP from the host network namespace when kube-proxy is started in IPVS mode without either masqueradeAll or clusterCIDR flags (#65388, @lbernail)
• checkLimitsForResolvConf for the pod create and update events instead of checking period (#64860, @wgliang)
• Introduce ContainersReady condition in Pod Status (#64646, @freehan)
• Declare IPVS-based kube-proxy GA (#58442, @m1093782566)
• Use Patch instead of Put to sync pod status (#62306, @freehan)
• Report node DNS info with --node-ip flag (#63170, @micahhausler)
• Re-enable nodeipam controller for external clouds. (#63049, @andrewsykim)
• NetworkPolicies can now target specific pods in other namespaces by including both a namespaceSelector and a podSelector in the same peer element. (#60452, @danwinship)

SIG Node

• User can now use sudo crictl on GCE cluster. (#65389, @Random-Liu)
• Fix concurrent map access panic (#65334, @dashpole)
• Add e2e regression tests for the kubelet being secure (#64140, @dixudx)
• ignore not found file error when watching manifests (#64880, @dixudx)
• Update crictl on GCE to v1.11.0. (#65254, @Random-Liu)
• Kubernetes depends on v0.30.1 of cAdvisor (#64987, @dashpole)
• deprecated and inactive option '--enable-custom-metrics' is removed in 1.11 (#60699, @CaoShuFeng)
• Setup dns servers and search domains for Windows Pods in dockershim. Docker EE version >= 17.10.0 is required for propagating DNS to containers. (#63905, @feiskyer)
• Use IONice to reduce IO priority of du and find (#64800, @dashpole)
• Kubelet will set extended resource capacity to zero after it restarts. If the extended resource is exported by a device plugin, its capacity will change to a valid value after the device plugin re-connects with the Kubelet. If the extended resource is exported by an external component through direct node status capacity patching, the component should repatch the field after kubelet becomes ready again. During the time gap, pods previously assigned with such resources may fail kubelet admission but their controller should create new pods in response to such failures. (#64784, @jiayingz)
• The Sysctls experimental feature has been promoted to beta (enabled by default via the Sysctls feature flag). PodSecurityPolicy and Pod objects now have fields for specifying and controlling sysctls. Alpha sysctl annotations will be ignored by 1.11+ kubelets. All alpha sysctl annotations in existing deployments must be converted to API fields to be effective. (#63717, @ingvagabund)
• Kubelet: Add security context for Windows containers (#64009, @feiskyer)
• Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta (#63724, @ravisantoshgudimetla)
• Add log and fs stats for Windows containers (#62266, @feiskyer)
• Ability to quota resources by priority (#57963, @vikaschoudhary16)
• Adds feature gate for plugin watcher (#64605, @vikaschoudhary16)
• Kubelet now proxies container streaming between apiserver and container runtime. The connection between kubelet and apiserver is authenticated. Container runtime should change streaming server to serve on localhost, to make the connection between kubelet and container runtime local. (#64006, @Random-Liu)
• Use Patch instead of Put to sync pod status (#62306, @freehan)
• Adds the VolumeSubpathEnvExpansion alpha feature to support environment variable expansion (#49388, @kevtaylor)
• Add probe based mechanism for kubelet plugin discovery (#63328, @vikaschoudhary16)
• Fail fast if cgroups-per-qos is set on Windows (#62984, @feiskyer)
• The dynamic Kubelet config feature is now beta, and the DynamicKubeletConfig feature gate is on by default. In order to use dynamic Kubelet config, ensure that the Kubelet's --dynamic-config-dir option is set. (#64275, @mtaufen)
• The Kubelet now exports metrics that report the assigned (node_config_assigned), last-known-good (node_config_last_known_good), and active (node_config_active) config sources, and a metric indicating whether the node is experiencing a config-related error (node_config_error). The config source metrics always report the value 1, and carry the node_config_name, node_config_uid, node_config_resource_version, and node_config_kubelet_key labels, which identify the config version. The error metric reports 1 if there is an error, 0 otherwise. (#57527, @mtaufen)
• The kubelet certificate rotation feature can now be enabled via the .RotateCertificates field in the kubelet's config file. The --rotate-certificates flag is now deprecated, and will be removed in a future release. (#63912, @luxas)
• The dynamic Kubelet config feature will now update config in the event of a ConfigMap mutation, which reduces the chance for silent config skew. Only name, namespace, and kubeletConfigKey may now be set in Node.Spec.ConfigSource.ConfigMap. The least disruptive pattern for config management is still to create a new ConfigMap and incrementally roll out a new Node.Spec.ConfigSource. (#63221, @mtaufen)
• [action required] The formerly publicly-available cAdvisor web UI that the kubelet ran on port 4194 by default is now turned off by default. The flag configuring what port to run this UI on --cadvisor-port was deprecated in v1.10. Now the default is --cadvisor-port=0, in other words, to not run the web server. The recommended way to run cAdvisor if you still need it, is via a DaemonSet. The --cadvisor-port will be removed in v1.12 (#63881, @luxas)
• Fix memory cgroup notifications, and reduce associated log spam. (#63220, @dashpole)
• Kubernetes cluster on GCE have crictl installed now. Users can use it to help debug their node. The documentation of crictl can be found https://github.com/kubernetes-incubator/cri-tools/blob/master/docs/crictl.md. (#63357, @Random-Liu)
• The NodeRestriction admission plugin now prevents kubelets from modifying/removing taints applied to their Node API object. (#63167, @liggitt)
• The status of dynamic Kubelet config is now reported via Node.Status.Config, rather than the KubeletConfigOk node condition. (#63314, @mtaufen)
• kubelet: fix hangs in updating Node status after network interruptions/changes between the kubelet and API server (#63492, @liggitt)
• Report node DNS info with --node-ip flag (#63170, @micahhausler)
• Re-enable nodeipam controller for external clouds. (#63049, @andrewsykim)
• The Kubelet's deprecated --allow-privileged flag now defaults to true. This enables users to stop setting --allow-privileged in order to transition to PodSecurityPolicy. Previously, users had to continue setting --allow-privileged, because the default was false. (#63442, @mtaufen)
• You must now specify Node.Spec.ConfigSource.ConfigMap.KubeletConfigKey when using dynamic Kubelet config to tell the Kubelet which key of the ConfigMap identifies its config file. (#59847, @mtaufen)
• kube-apiserver: --endpoint-reconciler-type now defaults to lease. The master-count reconciler is deprecated and will be removed in 1.13. (#58474, @rphillips)
• add warnings on using pod-infra-container-image for remote container runtime (#62982, @dixudx)
• kubelets are no longer allowed to delete their own Node API object. Prior to 1.11, in rare circumstances related to cloudprovider node ID changes, kubelets would attempt to delete/recreate their Node object at startup. If a legacy kubelet encounters this situation, a cluster admin can remove the Node object: (#62818, @mikedanese)
• fix resultRun by resetting it to 0 on pod restart (#62853, @tony612)
• The --experimental-qos-reserve kubelet flags is replaced by the alpha level --qos-reserved flag or QOSReserved field in the kubeletconfig and requires the QOSReserved feature gate to be enabled. (#62509, @sjenning)
• cluster/kube-up.sh now provisions a Kubelet config file for GCE via the metadata server. This file is installed by the corresponding GCE init scripts. (#62183, @mtaufen)
• Default mount propagation has changed from "HostToContainer" ("rslave" in Linux terminology) to "None" ("private") to match the behavior in 1.9 and earlier releases. "HostToContainer" as a default caused regressions in some pods. (#62462, @jsafrane)
• The node authorizer now automatically sets up rules for Node.Spec.ConfigSource when the DynamicKubeletConfig feature gate is enabled. (#60100, @mtaufen)
• CRI: define the mount behavior when host path does not exist: runtime should report error if the host path doesn't exist (#61460, @feiskyer)

SIG OpenStack

• in-tree support for openstack credentials is now deprecated. please use the "client-keystone-auth" from the cloud-provider-openstack repository. details on how to use this new capability is documented here - https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-client-keystone-auth.md (#64346, @dims)
• Provide a meaningful error message in openstack cloud provider when no valid IP address can be found for a node (#64318, @gonzolino)
• Restores the pre-1.10 behavior of the openstack cloud provider which uses the instance name as the Kubernetes Node name. This requires instances be named with RFC-1123 compatible names. (#63903, @liggitt)
• read openstack auth config from client config (#60200, @dixudx)

SIG Release

• Update to use go1.10.1 (#60597, @cblecker)

SIG Scalability

• Fix scheduler informers to receive events for all the pods in the cluster. (#63003, @bsalamat)

SIG Scheduling

• fixes a regression in kube-scheduler to properly load client connection information from a --config file that references a kubeconfig file (#65507, @liggitt)
• Update Rescheduler's manifest to use version 0.4.0. (#65454, @bsalamat)
• Improve scheduler's performance by eliminating sorting of nodes by their score. (#65396, @bsalamat)
• Add more conditions to the list of predicate failures that won't be resolved by preemption. (#64995, @bsalamat)
• fixes a memory leak in the kube-controller-manager observed when large numbers of pods with tolerations are created/deleted (#65339, @liggitt)
• Split 'scheduling_latency_seconds' metric into finer steps (predicate, priority, premption) (#65306, @shyamjvs)
• This PR will leverage subtests on the existing table tests for the scheduler units. (#63662, @xchapter7x)
• This PR will leverage subtests on the existing table tests for the scheduler units. (#63661, @xchapter7x)
• This PR will leverage subtests on the existing table tests for the scheduler units. (#63660, @xchapter7x)
• keep pod state consistent when scheduler cache UpdatePod (#64692, @adohe)
• This PR will leverage subtests on the existing table tests for the scheduler units. (#63659, @xchapter7x)
• Volume topology aware dynamic provisioning (#63193, @lichuqiang)
• API change for volume topology aware dynamic provisioning (#63233, @lichuqiang)
• Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta (#63724, @ravisantoshgudimetla)
• Revert #64364 to resurrect rescheduler. More info kubernetes/kubernetes#64725 :) (#64592, @ravisantoshgudimetla)
• Ability to quota resources by priority (#57963, @vikaschoudhary16)
• Add Alpha support for dynamic volume limits based on node type (#64154, @gnufied)
• ScheduleDaemonSetPods is an alpha feature (since v1.11) that causes DaemonSet Pods (#63223, @k82cn)
• Remove rescheduler from master. (#64364, @ravisantoshgudimetla)
• Basic plumbing for volume topology aware dynamic provisioning (#63232, @lichuqiang)
• None (#63975, @godliness)
• minor fix for VolumeZoneChecker predicate, storageclass can be in annotation and spec. (#63749, @wenlxie)
• The NodeRestriction admission plugin now prevents kubelets from modifying/removing taints applied to their Node API object. (#63167, @liggitt)
• This PR will leverage subtests on the existing table tests for the scheduler units. (#63658, @xchapter7x)
• The PriorityClass API is promoted to scheduling.k8s.io/v1beta1 (#63100, @ravisantoshgudimetla)
• You must now specify Node.Spec.ConfigSource.ConfigMap.KubeletConfigKey when using dynamic Kubelet config to tell the Kubelet which key of the ConfigMap identifies its config file. (#59847, @mtaufen)
• Supported nodeSelector.matchFields (node's metadata.node) in scheduler. (#62453, @k82cn)
• Increase scheduler cache generation number monotonically in order to avoid collision and use of stale information in scheduler. (#63264, @bsalamat)
• Added CheckNodePIDPressurePredicate to checks if a pod can be scheduled on (#60007, @k82cn)
• Added MatchFields to NodeSelectorTerm; in 1.11, it only support metadata.name. (#62002, @k82cn)
• Fix scheduler informers to receive events for all the pods in the cluster. (#63003, @bsalamat)
• Split PodPriority and PodPreemption feature gate (#62243, @resouer)
• Add write-config-to to scheduler (#62515, @resouer)
• Fix inter-pod anti-affinity check to consider a pod a match when all the anti-affinity terms match. (#62715, @bsalamat)
• Pod affinity nodeSelectorTerm.matchExpressions may now be empty, and works as previously documented: nil or empty matchExpressions matches no objects in scheduler. (#62448, @k82cn)
• Fix an issue in inter-pod affinity predicate that cause affinity to self being processed incorrectly (#62591, @bsalamat)
• The kubeadm config option API.ControlPlaneEndpoint has been extended to take an optional port which may differ from the apiserver's bind port. (#62314, @rjosephwright)
• improve performance of affinity/anti-affinity predicate of default scheduler significantly. (#62211, @bsalamat)
• Schedule even if extender is not available when using extender (#61445, @resouer)
• Pods requesting resources prefixed with *kubernetes.io will remain unscheduled if there are no nodes exposing that resource. (#61860, @mindprince)
• Balanced resource allocation priority in scheduler to include volume count on node (#60525, @ravisantoshgudimetla)
• kube-scheduler has been fixed to use --leader-elect option back to true (as it was in previous versions) (#59732, @dims)
• Use inline func to ensure unlock is executed (#61644, @resouer)
• Automatically add system critical priority classes at cluster boostrapping. (#60519, @bsalamat)

SIG Storage

• AWS now checks for validity of ecryption key when creating encrypted volumes. Dynamic provisioning of encrypted volume may get slower due to these checks. (#65223, @jsafrane)
• Fixes an issue where Portworx PVCs remain in pending state when created using a StorageClass with empty parameters (#64895, @harsh-px)
• Added block volume support to Cinder volume plugin. (#64879, @bertinatto)
• Provides API support for external CSI storage drivers to support block volumes. (#64723, @vladimirvivien)
• Volume topology aware dynamic provisioning (#63193, @lichuqiang)
• Updated Container Storage Interface specification version to v0.3.0 (#64719, @davidz627)
• Add a volume projection that is able to project service account tokens. (#62005, @mikedanese)
• PodSecurityPolicy now supports restricting hostPath volume mounts to be readOnly and under specific path prefixes (#58647, @jhorwit2)
• API change for volume topology aware dynamic provisioning (#63233, @lichuqiang)
• Adding CSI driver registration code. (#64560, @sbezverk)
• Fixes issue for readOnly subpath mounts for SELinux systems and when the volume mountPath already existed in the container image. (#64351, @msau42)
• Implement kubelet side online file system resizing (#62460, @mlmhl)
• AWS EBS volumes can be now used as ReadOnly in pods. (#64403, @jsafrane)
• Adds the VolumeSubpathEnvExpansion alpha feature to support environment variable expansion (#49388, @kevtaylor)
• (ACTION REQUIRED) PersisntVolumeLabel admission controller is now disabled by default. If you depend on this feature (AWS/GCE) then ensure it is added to the --enable-admission-plugins flag on the kube-apiserver. (#64326, @andrewsykim)
• Move Volume expansion to Beta (#64288, @gnufied)
• Basic plumbing for volume topology aware dynamic provisioning (#63232, @lichuqiang)
• Fixes issue where subpath readOnly mounts failed (#63045, @msau42)
• StorageObjectInUseProtection feature is GA. (#62870, @pospispa)
• Add support to resize Portworx volumes. (#62308, @harsh-px)
• Fix user visible files creation for windows (#62375, @feiskyer)
• gitRepo volumes in pods no longer require git 1.8.5 or newer, older git versions are supported too now. (#62394, @jsafrane)
• Default mount propagation has changed from "HostToContainer" ("rslave" in Linux terminology) to "None" ("private") to match the behavior in 1.9 and earlier releases. "HostToContainer" as a default caused regressions in some pods. (#62462, @jsafrane)
• fix incompatible file type checking on Windows (#62154, @dixudx)
• Fixes bug in rbd-nbd utility when rbd is used. (#62168, @piontec)
• CRI: define the mount behavior when host path does not exist: runtime should report error if the host path doesn't exist (#61460, @feiskyer)
• Fix mounting of UNIX sockets(and other special files) in subpaths (#61480, @gnufied)

SIG Testing

• Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta (#63724, @ravisantoshgudimetla)
• kube-apiserver: --endpoint-reconciler-type now defaults to lease. The master-count reconciler is deprecated and will be removed in 1.13. (#58474, @rphillips)

SIG VMWare

• In the vSphere cloud provider the Global.vm-uuid configuration option is not deprecated anymore, it can be used to overwrite the VMUUID on the controller-manager (#65152, @alvaroaleman)
• vSphere Cloud Provider: add SAML token authentication support (#63824, @dougm)

SIG Windows

• Updated hcsshim dependency to v0.6.11 (#64272, @jessfraz)
• Setup dns servers and search domains for Windows Pods in dockershim. Docker EE version >= 17.10.0 is required for propagating DNS to containers. (#63905, @feiskyer)
• Kubelet: Add security context for Windows containers (#64009, @feiskyer)
• Add log and fs stats for Windows containers (#62266, @feiskyer)
• Fail fast if cgroups-per-qos is set on Windows (#62984, @feiskyer)
• Fix user visible files creation for windows (#62375, @feiskyer)
• fix incompatible file type checking on Windows (#62154, @dixudx)

Bug Fixes

• fixes a regression in kube-scheduler to properly load client connection information from a --config file that references a kubeconfig file (#65507, @liggitt)
• Allow access to ClusterIP from the host network namespace when kube-proxy is started in IPVS mode without either masqueradeAll or clusterCIDR flags (#65388, @lbernail)
• User can now use sudo crictl on GCE cluster. (#65389, @Random-Liu)
• fixes a memory leak in the kube-controller-manager observed when large numbers of pods with tolerations are created/deleted (#65339, @liggitt)
• Fix concurrent map access panic (#65334, @dashpole)
• Split 'scheduling_latency_seconds' metric into finer steps (predicate, priority, premption) (#65306, @shyamjvs)
• Fixes an issue where Portworx PVCs remain in pending state when created using a StorageClass with empty parameters (#64895, @harsh-px)
• fixed incorrect OpenAPI schema for CustomResourceDefinition objects (#65256, @liggitt)
• ignore not found file error when watching manifests (#64880, @dixudx)
• Fixes missing nodes lines when kubectl top nodes (#64389, @yue9944882)
• Updates Cluster Autoscaler version to 1.3.0. Release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.3.0 (#65219, @aleksandra-malinowska)
• The deprecated --service-account-private-key-file flag has been removed from the cloud-controller-manager. The flag is still present and supported in the kube-controller-manager. (#65182, @liggitt)
• Update Cluster Autoscaler to v1.3.0-beta.2. Release notes for this version: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.3.0-beta.2 (#65148, @aleksandra-malinowska)
• GCE: Fixes operation polling to adhere to the specified interval. Furthermore, operation errors are now returned instead of ignored. (#64630, @nicksardo)
• Updated hcsshim dependency to v0.6.11 (#64272, @jessfraz)
• kubeadm - Ensure the peer port is secured by explicitly setting the peer URLs for the default etcd instance. (#64988, @detiber)
• Kubernetes depends on v0.30.1 of cAdvisor (#64987, @dashpole)
• A cluster-autoscaler ClusterRole is added to cover only the functionality required by Cluster Autoscaler and avoid abusing system:cluster-admin role. (#64503, @kgolab)
• Webhooks for the mutating admission controller now support "remove" operation. (#64255, @rojkov)
• kubernetes now packages cri-tools (crictl) in addition to all the other kubeadm tools in a deb and rpm. (#64836, @chuckha)
• Setup dns servers and search domains for Windows Pods in dockershim. Docker EE version >= 17.10.0 is required for propagating DNS to containers. (#63905, @feiskyer)
• kubeadm: only run kube-proxy on architecture consistent nodes (#64696, @dixudx)
• Orphan delete is now supported for custom resources (#63386, @roycaihw)
• kubectl will list all allowed print formats when an invalid format is passed. (#64371, @CaoShuFeng)
• Use IONice to reduce IO priority of du and find (#64800, @dashpole)
• Kubelet will set extended resource capacity to zero after it restarts. If the extended resource is exported by a device plugin, its capacity will change to a valid value after the device plugin re-connects with the Kubelet. If the extended resource is exported by an external component through direct node status capacity patching, the component should repatch the field after kubelet becomes ready again. During the time gap, pods previously assigned with such resources may fail kubelet admission but their controller should create new pods in response to such failures. (#64784, @jiayingz)
• kubeadm now configures the etcd liveness probe correctly when etcd is listening on all interfaces (#64670, @stealthybox)
• Kubelet: Add security context for Windows containers (#64009, @feiskyer)
• Fix kubectl auth can-i exit code. It will return 1 if the user is not allowed and 0 if it's allowed. (#59579, @fbac)
• apply global flag "context" for kubectl config view --minify (#64608, @dixudx)
• Fix kube-controller-manager panic while provisioning Azure security group rules (#64739, @feiskyer)
• fixes a potential deadlock in the garbage collection controller (#64235, @liggitt)
• Fixes issue for readOnly subpath mounts for SELinux systems and when the volume mountPath already existed in the container image. (#64351, @msau42)
• Add log and fs stats for Windows containers (#62266, @feiskyer)
• Meta data of CustomResources is now pruned and schema checked during deserialization of requests and when read from etcd. In the former case, invalid meta data is rejected, in the later it is dropped from the CustomResource objects. (#64267, @sttts)
• Adds feature gate for plugin watcher (#64605, @vikaschoudhary16)
• kubeadm now preserves previous manifests after upgrades (#64337, @liztio)
• kubeadm: conditionally set the kubelet cgroup driver for Docker (#64347, @neolit123)
• Label ExternalEtcdClientCertificates can be used for ignoring all preflight check issues related to client certificate files for external etcd. (#64269, @kad)
• Provide a meaningful error message in openstack cloud provider when no valid IP address can be found for a node (#64318, @gonzolino)
• kubeadm: rename the addon parameter kube-dns to coredns for kubeadm alpha phases addons as CoreDNS is now the default DNS server in 1.11. (#64274, @neolit123)
• Fix SessionAffinity not updated issue for Azure load balancer (#64180, @feiskyer)
• event object references with apiversion will now report an apiversion. (#63913, @deads2k)
• Cluster Autoscaler 1.2.2 (release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.2.2) (#63974, @aleksandra-malinowska)
• Fix memory cgroup notifications, and reduce associated log spam. (#63220, @dashpole)
• Restores the pre-1.10 behavior of the openstack cloud provider which uses the instance name as the Kubernetes Node name. This requires instances be named with RFC-1123 compatible names. (#63903, @liggitt)
• kubelet: fix hangs in updating Node status after network interruptions/changes between the kubelet and API server (#63492, @liggitt)
• corrects a race condition in bootstrapping aggregated cluster roles in new HA clusters (#63761, @liggitt)
• The Kubelet's deprecated --allow-privileged flag now defaults to true. This enables users to stop setting --allow-privileged in order to transition to PodSecurityPolicy. Previously, users had to continue setting --allow-privileged, because the default was false. (#63442, @mtaufen)
• Fixed where we get latest builds for stable branches (#63504, @dims)
• When updating /status subresource of a custom resource, only the value at the .status subpath for the update is considered. (#63385, @CaoShuFeng)
• Do not check vmSetName when getting Azure node's IP (#63541, @feiskyer)
• Azure VMSS: support VM names to contain the _ character (#63526, @djsly)
• Increase scheduler cache generation number monotonically in order to avoid collision and use of stale information in scheduler. (#63264, @bsalamat)
• KUBE_API_VERSIONS is no longer respected. It was used for testing, but runtime-config is the proper flag to set. (#63165, @deads2k)
• fix resultRun by resetting it to 0 on pod restart (#62853, @tony612)
• Introduce truncating audit backend that can be enabled for existing backend to limit the size of individual audit events and batches of events. (#61711, @crassirostris)
• kubeadm upgrade no longer races leading to unexpected upgrade behavior on pod restarts (#62655, @stealthybox)
• removed unsafe double RLock in cpumanager (#62464, @choury)
• stop kubelet to cloud provider integration potentially wedging kubelet sync loop (#62543, @ingvagabund)
• kubectl stops rendering List as suffix kind name for CRD resources (#62512, @dixudx)
• Pod affinity nodeSelectorTerm.matchExpressions may now be empty, and works as previously documented: nil or empty matchExpressions matches no objects in scheduler. (#62448, @k82cn)
• Fix user visible files creation for windows (#62375, @feiskyer)
• Ensure expected load balancer is selected for Azure (#62450, @feiskyer)
• Resolves forbidden error when the daemon-set-controller cluster role access controllerrevisions resources. (#62146, @frodenas)
• Fixed #731 kubeadm upgrade ignores HighAvailability feature gate (#62455, @fabriziopandini)
• Cluster Autoscaler 1.2.1 (release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.2.1) (#62457, @mwielgus)
• kubectl: improves compatibility with older servers when creating/updating API objects (#61949, @liggitt)
• kubectl: restore the ability to show resource kinds when displaying multiple objects (#61985, @liggitt)
• Ensure reasons end up as comments in kubectl edit. (#60990, @bmcstdio)
• kubectl: fixes issue with -o yaml and -o json omitting kind and apiVersion when used with --dry-run (#61808, @liggitt)
• [fluentd-gcp addon] Partial success option is enabled in fluentd. (#61773, @crassirostris)
• Fix racy panics when using fake watches with ObjectTracker (#61195, @grantr)
• Ensure cloudprovider.InstanceNotFound is reported when the VM is not found on Azure (#61531, @feiskyer)
• escape literal percent sign when formatting (#61523, @dixudx)
• fix sorting taints in case the sorting keys are equal (#61255, @dixudx)

Other Notable Changes

• kubectl convert previous created a list inside of a list. Now it is only wrapped once. (#65489, @deads2k)
• Fixed cleanup of CSI metadata files. (#65323, @jsafrane)
• Tolerate missing watch permission when deleting a resource (#65370, @deads2k)
• Prevents a kubectl delete hang when deleting controller managed lists (#65367, @deads2k)
• Change Azure ARM Rate limiting error message. (#65292, @wgliang)
• Report accurate status for kubernetes-master and -worker charms. (#65187, @kwmonroe)
• fluentd-gcp grace termination period increased to 60s. (#65084, @x13n)
• Pass cluster_location argument to Heapster (#65176, @kawych)
• Corrected a mistake in the documentation for wait.PollImmediate(...) (#65026, @spew)
• Updated default image for nginx ingress in CDK to match current Kubernetes docs. (#64285, @hyperbolic2346)
• add port-forward examples for sevice (#64773, @MasayaAoyama)
• Increase the gRPC max message size to 16MB in the remote container runtime. (#64672, @mcluseau)
• The new default value for the --allow-privileged parameter of the Kubernetes-worker charm has been set to true based on changes which went into the Kubernetes 1.10 release. Before this change the default value was set to false. If you're installing Canonical Kubernetes you should expect this value to now be true by default and you should now look to use PSP (pod security policies). (#64104, @CalvinHartwell)
• The --remove-extra-subjects and --remove-extra-permissions flags have been enabled for kubectl auth reconcile (#64541, @mrogers950)
• kubectl built for darwin from darwin now enables cgo to use the system-native C libraries for DNS resolution. Cross-compiled kubectl (e.g. from an official kubernetes release) still uses the go-native netgo DNS implementation. (#64219, @ixdy)
• kubectl apply --prune supports CronJob resource. (#62991, @tomoe)
• Fix issue of colliding nodePorts when the cluster has services with externalTrafficPolicy=Local (#64349, @nicksardo)
• fixes a panic applying json patches containing out of bounds operations (#64355, @liggitt)
• kubectl delete does not use reapers for removing objects anymore, but relies on server-side GC entirely (#63979, @soltysh)
• Changes ext3/ext4 volume creation to not reserve any portion of the volume for the root user. (#64102, @atombender)
• adds a kubectl wait command (#64034, @deads2k)
• Graduate CRI container log rotation to beta, and enable it by default. (#64046, @yujuhong)
• APIServices with kube-like versions (e.g. v1, v2beta1, etc.) will be sorted appropriately within each group. (#64004, @mbohlool)
• increase grpc client default response size (#63977, @runcom)
• Adds a mechanism in vSphere Cloud Provider to get credentials from Kubernetes secrets (#63902, @abrarshivani)
• Update event-exporter to version v0.2.0 that supports old (gke_container/gce_instance) and new (k8s_container/k8s_node/k8s_pod) stackdriver resources. (#63918, @cezarygerard)
• Add 'UpdateStrategyType' and 'RollingUpdateStrategy' to 'kubectl describe sts' command output. (#63844, @tossmilestone)
• Remove UID mutation from request.context. (#63957, @hzxuzhonghu)
• Add a way to pass extra arguments to etcd. (#63961, @mborsz)
• Added support for NFS relations on kubernetes-worker charm. (#63817, @hyperbolic2346)
• CRI: update documents for container logpath. The container log path has been changed from containername_attempt#.log to containername/attempt#.log (#62015, @feiskyer)
• Create a new dryRun query parameter for mutating endpoints. If the parameter is set, then the query will be rejected, as the feature is not implemented yet. This will allow forward compatibility with future clients; otherwise, future clients talking with older apiservers might end up modifying a resource even if they include the dryRun query parameter. (#63557, @apelisse)
• Services can listen on same host ports on different interfaces with --nodeport-addresses specified (#62003, @m1093782566)
• Adding initial Korean translation for kubectl (#62040, @ianychoi)
• Apply pod name and namespace labels to pod cgroup in cAdvisor metrics (#63406, @derekwaynecarr)
• Include the list of security groups when failing with the errors that more then one is tagged (#58874, @sorenmat)
• Allow "required" to be used at the CRD OpenAPI validation schema when the /status subresource is enabled. (#63533, @sttts)
• Bump Heapster to v1.5.3 (#63535, @serathius)
• [fluentd-gcp addon] Use the logging agent's node name as the metadata agent URL. (#63353, @bmoyles0117)
• kubectl cp supports completion. (#60371, @superbrothers)
• OpenStack built-in cloud provider is now deprecated. Please use the external cloud provider for OpenStack. (#63524, @dims)
• kubectl delete with selection criteria defaults to ignoring not found errors (#63490, @deads2k)
• Update to use go1.10.2 (#63412, @praseodym)
• kube-apiserver: the default --endpoint-reconciler-type is now lease. The master-count endpoint reconciler type is deprecated and will be removed in 1.13. (#63383, @liggitt)
• Use /usr/bin/env in all script shebangs to increase portability. (#62657, @matthyx)
• OpenStack cloudprovider: Fix deletion of orphaned routes (#62729, @databus23)
• Fix a bug that headless service without ports fails to have endpoint created. (#62497, @MrHohn)
• Fix panic for attaching AzureDisk to vmss nodes (#63275, @feiskyer)
• fix cephfs fuse mount bug when user is not admin (#61804, @zhangxiaoyu-zidif)
• fixed spurious "unable to find api field" errors patching custom resources (#63146, @liggitt)
• Remove METADATA_AGENT_VERSION configuration option. (#63000, @kawych)
• Fix the liveness probe to use /bin/bash -c instead of /bin/bash c. (#63033, @bmoyles0117)
• The --bootstrap-kubeconfig argument to Kubelet previously created the first bootstrap client credentials in the certificates directory as kubelet-client.key and kubelet-client.crt. Subsequent certificates created by cert rotation were created in a combined PEM file that was atomically rotated as kubelet-client-DATE.pem in that directory, which meant clients relying on the node.kubeconfig generated by bootstrapping would never use a rotated cert. The initial bootstrap certificate is now generated into the cert directory as a PEM file and symlinked to kubelet-client-current.pem so that the generated kubeconfig remains valid after rotation. (#62152, @smarterclayton)
• Fix error where config map for Metadata Agent was not created by addon manager. (#62909, @kawych)
• Fixes the kubernetes.default.svc loopback service resolution to use a loopback configuration. (#62649, @liggitt)
• fix permissions to allow statefulset scaling for admins, editors, and viewers (#62336, @deads2k)
• GCE: Fix for internal load balancer management resulting in backend services with outdated instance group links. (#62885, @nicksardo)
• Set pod status to "Running" if there is at least one container still reporting as "Running" status and others are "Completed". (#62642, @ceshihao)
• "NONE" (#58740, @YuxiJin-tobeyjin)
• Make volume usage metrics available for Cinder (#62668, @zetaab)
• Removes --include-extended-apis which was deprecated back in kubernetes/kubernetes#32894 (#62803, @deads2k)
• fix network setup in hack/local-up-cluster.sh (kubernetes/kubernetes#60431) (#60633, @pohly)
• GCE: Bump GLBC version to 1.1.1 - fixing an issue of handling multiple certs with identical certificates (#62751, @nicksardo)
• Allow user to scale l7 default backend deployment (#62685, @freehan)
• Add @andrewsykim as an approver for CCM related code. (#62749, @andrewsykim)
• Update kube-dns to Version 1.14.10. Major changes: (#62676, @MrHohn)
• remove deprecated initresource admission plugin (#58784, @wackxu)
• Fix machineID getting for vmss nodes when using instance metadata (#62611, @feiskyer)
• Fixes issue where PersistentVolume.NodeAffinity.NodeSelectorTerms were ANDed instead of ORed. (#62556, @msau42)
• Fix Forward chain default reject policy for IPVS proxier (#62007, @m1093782566)
• [fluentd-gcp addon] Increase CPU limit for fluentd to 1 core to achieve 100kb/s throughput. (#62430, @bmoyles0117)
• GCE: Bump GLBC version to 1.1.0 - supporting multiple certificates and HTTP2 (#62427, @nicksardo)
• Add generators for apps/v1 deployments. (#61288, @ayushpateria)
• Allow additionalProperties in CRD OpenAPI v3 specification for validation, mutually exclusive to properties. (#62333, @sttts)
• cinder volume plugin : (#61082, @wenlxie)
• "NONE" (#59035, @YuxiJin-tobeyjin)
• RBAC information is included in audit logs via audit.Event annotations: (#58807, @CaoShuFeng)
• Update kube-dns to Version 1.14.9 in kubeadm. (#61918, @MrHohn)
• Add support to ingest log entries to Stackdriver against new "k8s_container" and "k8s_node" resources. (#62076, @qingling128)
• remove deprecated --mode flag in check-network-mode (#60102, @satyasm)
• Add support to ingest service level metadata against the "k8s_cluster" resource. (#62043, @supriyagarg)
• Disabled CheckNodeMemoryPressure and CheckNodeDiskPressure predicates if TaintNodesByCondition enabled (#60398, @k82cn)
• kubeadm config can now override the Node CIDR Mask Size passed to kube-controller-manager. (#61705, @jstangroome)
• Support custom test configuration for IPAM performance integration tests (#61959, @satyasm)
• GCE: Updates GLBC version to 1.0.1 which includes a fix which prevents multi-cluster ingress objects from creating full load balancers. (#62075, @nicksardo)
• Rev the Azure SDK for networking to 2017-06-01 (#61955, @brendandburns)
• kubernetes-master charm now supports metrics server for horizontal pod autoscaler. (#60174, @hyperbolic2346)
• fix scheduling policy on ConfigMap breaks without the --policy-configmap-namespace flag set (#61388, @zjj2wry)
• kubectl certificate approve|deny will not modify an already approved or denied CSR unless the --force flag is provided. (#61971, @smarterclayton)
• Kubelet now exposes a new endpoint /metrics/probes which exposes a Prometheus metric containing the liveness and/or readiness probe results for a container. (#61369, @rramkumar1)
• new dhcp-domain parameter to be used for figuring out the hostname of a node (#61890, @dims)
• Update kube-dns to Version 1.14.9. Major changes: (#61908, @MrHohn)
• Fixed ingress issue with CDK and pre-1.9 versions of kubernetes. (#61859, @hyperbolic2346)
• Removed rknetes code, which was deprecated in 1.10. (#61432, @filbranden)
• Disable ipamperf integration tests as part of every PR verification. (#61863, @satyasm)
• Enable server-side print in kubectl by default, with the ability to turn it off with --server-print=false (#61477, @soltysh)
• In a GCE cluster, the default HAIRPIN_MODE is now "hairpin-veth". (#60166, @rramkumar1)
• Deployment will stop adding pod-template-hash labels/selector to ReplicaSets and Pods it adopts. Resources created by Deployments are not affected (will still have pod-template-hash labels/selector). (#61615, @janetkuo)
• Updated admission controller settings for Juju deployed Kubernetes clusters (#61427, @hyperbolic2346)
• Performance test framework and basic tests for the IPAM controller, to simulate behavior (#61143, @satyasm)
• kubernetes-master charm now properly clears the client-ca-file setting on the apiserver snap (#61479, @hyperbolic2346)
• `kubectl apply view/edit-last-applied support completion. (#60499, @superbrothers)
• Add all kinds of resource objects' statuses in HPA description. (#59609, @zhangxiaoyu-zidif)
• Bound cloud allocator to 10 retries with 100 ms delay between retries. (#61375, @satyasm)
• Removed always pull policy from the template for ingress on CDK. (#61598, @hyperbolic2346)