saschagrunert/v1.17.0-by-kind-3.md

## v1.17.0-by-kind-3.md

      
    Raw
  

              v1.17.0-by-kind-3.md
            
          
    Changes by Kind

API Change


Deprecate the beta labels for zones ("failure-domain.beta.kubernetes.io/zone") and
regions ("failure-domain.beta.kubernetes.io/region") in favor of their GA equivalents:
"topology.kubernetes.io/zone" and "topology.kubernetes.io/region".
The beta labels "failure-domain.beta.kubernetes.io/zone" and "failure-domain.beta.kubernetes.io/region" will be removed in v1.21 (#81431, @andrewsykim) [SIG Apps, SIG Auth, SIG Cloud Provider, SIG Network, SIG Node, SIG Scheduling, SIG Storage, and SIG Testing]


A new kubelet command line option, --reserved-cpus, is introduced to explicitly define the the CPU list that will be reserved for system. For example, if --reserved-cpus=0,1,2,3 is specified, then cpu 0,1,2,3 will be reserved for the system.  On a system with 24 CPUs, the user may specify isolcpus=4-23 for the kernel option and use CPU 4-23 for the user containers. (#83592, @jianzzha) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Node]


Moving WindowsRunAsUserName feature to beta (#84882, @marosset) [SIG Apps, SIG Node, and SIG Windows]


Deprecate the instance type beta label ("beta.kubernetes.io/instance-type") in favor of it's GA equivalent: "node.kubernetes.io/instance-type" (#82049, @andrewsykim) [SIG Apps, SIG Auth, SIG Cloud Provider, SIG Node, and SIG Scheduling]


kube-controller-manager
--node-cidr-mask-size-ipv4 int32     Default: 24. Mask size for IPv4 node-cidr in dual-stack cluster.
--node-cidr-mask-size-ipv6 int32     Default: 64. Mask size for IPv6 node-cidr in dual-stack cluster.
These 2 flags can be used only for dual-stack clusters. For non dual-stack clusters, continue to use
--node-cidr-mask-size flag to configure the mask size.
The default node cidr mask size for IPv6 was 24 which is now changed to 64. (#79993, @aramase) [SIG API Machinery, SIG Apps, SIG Cloud Provider, SIG Network, and SIG Testing]


Adds a new label to indicate what is managing an EndpointSlice. (#83965, @robscott) [SIG Apps, and SIG Network]


Added appProtocol field to EndpointSlice Port (#83815, @howardjohn) [SIG Apps, SIG Cluster Lifecycle, SIG Instrumentation, and SIG Network]


Adds Windows Server build information as a label on the node. (#84472, @gab-satchi) [SIG Node, and SIG Windows]


EndpointSlices are now beta and enabled by default for better Network Endpoint performance at scale. (#84390, @robscott) [SIG API Machinery, SIG Apps, SIG Auth, SIG CLI, SIG Network, and SIG Testing]


kube-apiserver: The AdmissionConfiguration type accepted by --admission-control-config-file has been promoted to apiserver.config.k8s.io/v1 with no schema changes. (#85098, @liggitt) [SIG API Machinery]


Splitting IP address type into IPv4 and IPv6 for EndpointSlices (#84971, @robscott) [SIG Apps, SIG CLI, SIG Network, SIG Scheduling, and SIG Testing]


Pod process namespace sharing is now Generally Available. The PodShareProcessNamespace feature gate is now deprecated and will be removed in Kubernetes 1.19. (#84356, @verb) [SIG Apps, SIG Node, and SIG Testing]


OpenAPI v3 format in CustomResourceDefinition schemas are now documented. (#85381, @sttts) [SIG API Machinery, and SIG Testing]


Implement the documented API semantics of list-type and map-type atomic to reject non-atomic sub-types. (#84722, @sttts) [SIG API Machinery]


Support Service Topology (#72046, @m1093782566) [SIG Apps, SIG Auth, and SIG Network]


action required


Currently, if users were to explicitly specify CacheSize of 0 for KMS provider, they would end-up with a provider that caches up to 1000 keys. This PR changes this behavior.
Post this PR, when users supply 0 for CacheSize this will result in a validation error.


CacheSize type was changed from int32 to *int32. This allows defaulting logic to differentiate between cases where users explicitly supplied 0 vs. not supplied any value.


KMS Provider's endpoint (path to Unix socket) is now validated when the EncryptionConfiguration files is loaded. This used to be handled by the GRPCService. (#85363, @immutableT) [SIG API Machinery, SIG Auth, and SIG Instrumentation]


CRDs defaulting is promoted to GA. Note: the feature gate CustomResourceDefaulting will be removed in 1.18. (#84713, @sttts) [SIG API Machinery, and SIG Testing]


All resources within the rbac.authorization.k8s.io/v1alpha1 and rbac.authorization.k8s.io/v1beta1 API groups are deprecated in favor of rbac.authorization.k8s.io/v1, and will no longer be served in v1.20. (#84758, @liggitt) [SIG Auth]


Scheduler ComponentConfig fields are now pointers (#83619, @damemi) [SIG Scheduling, and SIG Testing]


Fixed EndpointSlice port name validation to match Endpoint port name validation (allowing port names longer than 15 characters) (#84481, @robscott) [SIG Network]


Introduce x-kubernetes-map-type annotation as a CRD API extension. Enables this particular validation for server-side apply. (#84113, @enxebre) [SIG API Machinery]


Migrate controller-manager and scheduler to EndpointsLeases leader election. (#84084, @wojtek-t) [SIG API Machinery, SIG Apps, SIG Auth, SIG Cloud Provider, and SIG Scheduling]


Update etcd client side to v3.4.3
Deprecated prometheus request meta-metrics have been removed (http_request_duration_microseconds, http_request_duration_microseconds_sum, http_request_duration_microseconds_count, http_request_size_bytes, http_request_size_bytes_sum, http_request_size_bytes_count, http_requests_total, http_response_size_bytes, http_response_size_bytes_sum, http_response_size_bytes_count) due to removal from the prometheus client library. Prometheus http request meta-metrics are now generated from promhttp.InstrumentMetricHandler instead. (#83987, @wenjiaswe) [SIG API Machinery, SIG Apps, SIG Auth, SIG Autoscaling, SIG CLI, SIG Cloud Provider, SIG Cluster Lifecycle, SIG Instrumentation, SIG Network, SIG Node, SIG Scheduling, SIG Storage, and SIG Testing]


Scheduler Policy API has a new recommended apiVersion "apiVersion: kubescheduler.config.k8s.io/v1" which is consistent with the scheduler API group "kubescheduler.config.k8s.io". It holds the same API as the old apiVersion "apiVersion: v1". (#83578, @Huang-Wei) [SIG Scheduling, and SIG Testing]


CSI Topology feature is GA. The CSINodeInfo feature gate is deprecated and will be removed in a future release. The storage.k8s.io/v1beta1 CSINode object is deprecated and will be removed in a future release. (#83474, @msau42) [SIG API Machinery, SIG Apps, SIG Auth, SIG CLI, SIG Storage, and SIG Testing]


Adds FQDN addressType support for EndpointSlice. (#84091, @robscott) [SIG API Machinery, and SIG Network]


When scaling down a ReplicaSet, delete doubled up replicas first, where a "doubled up replica" is defined as one that is on the same node as an active replica belonging to a related ReplicaSet.  ReplicaSets are considered "related" if they have a common controller (typically a Deployment). (#80004, @Miciah) [SIG Apps, SIG Autoscaling, SIG Scalability, and SIG Testing]


Promote WatchBookmark feature to GA.
With WatchBookmark feature, clients are able to request watch events with BOOKMARK type. Clients should not assume bookmarks are returned at any specific interval, nor may they assume the server will send any BOOKMARK event during a session. (#83195, @wojtek-t) [SIG API Machinery]


An end-user may choose to request logs without confirming the identity of the backing kubelet.  This feature can be disabled by setting the AllowInsecureBackendProxy feature-gate to false. (#83419, @deads2k) [SIG Apps, SIG Node, and SIG Testing]


external facing APIs in pluginregistration and deviceplugin packages are now available under k8s.io/kubelet/pkg/apis/ (#83551, @dims) [SIG Node, and SIG Testing]


The VolumeSubpathEnvExpansion feature is graduating to GA. The VolumeSubpathEnvExpansion feature gate is unconditionally enabled, and will be removed in v1.19. (#82578, @kevtaylor) [SIG Apps, SIG Node, SIG Storage, and SIG Testing]


Add "podInitialBackoffDurationSeconds" and "podMaxBackoffDurationSeconds" to the scheduler config API (#81263, @draveness) [SIG Apps, and SIG Scheduling]


Fix typos in certificates.k8s.io/v1beta1 KeyUsage constant names: UsageContentCommittment becomes UsageContentCommitment and UsageNetscapSGC becomes UsageNetscapeSGC. (#82511, @abursavich) [SIG Auth, and SIG Cluster Lifecycle]


When registering with a 1.17+ API server, MutatingWebhookConfiguration and ValidatingWebhookConfiguration objects can now request that only v1 AdmissionReview requests be sent to them. Previously, webhooks were required to support receiving v1beta1 AdmissionReview requests as well for compatibility with API servers <= 1.15.

When registering with a 1.17+ API server, a CustomResourceDefinition conversion webhook can now request that only v1 ConversionReview requests be sent to them. Previously, conversion webhooks were required to support receiving v1beta1 ConversionReview requests as well for compatibility with API servers <= 1.15. (#82707, @liggitt) [SIG API Machinery]


Feature


Windows nodes on GCE can use TPM-based authentication to the master. (#85466, @pjh) [SIG Cluster Lifecycle]


kubectl/drain: add disable-eviction option.
Force drain to use delete, even if eviction is supported. This will bypass checking PodDisruptionBudgets, and should be used with caution. (#85571, @michaelgugino) [SIG CLI]


Updated EndpointSlices to use PublishNotReadyAddresses from Services. (#84573, @robscott) [SIG Apps, and SIG Network]


add RequiresExactMatch for label.Selector (#85048, @shaloulcy) [SIG API Machinery]


Deprecated metric rest_client_request_latency_seconds has been turned off. (#83836, @RainbowMango) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Instrumentation]


Removed dependency on kubectl from several storage E2E tests (#84042, @okartau) [SIG Storage, and SIG Testing]


Add plugin_execution_duration_seconds metric for scheduler framework plugins. (#84522, @liu-cong) [SIG Scheduling]


Node-specific volume limits has graduated to GA. (#83568, @bertinatto) [SIG Auth, SIG Node, SIG Scheduling, SIG Storage, and SIG Testing]


kubeadm: the command "kubeadm token create" now has a "--certificate-key" flag that can be used for the formation of join commands for control-planes with automatic copy of certificates (#84591, @TheLastProject) [SIG Cluster Lifecycle]


Promote NodeLease feature to GA.
The feature make Lease object changes an additional healthiness signal from Node. Together with that, we reduce frequency of NodeStatus updates to 5m by default in case of no changes to status itself (#84351, @wojtek-t) [SIG API Machinery, SIG Apps, SIG Auth, SIG Node, and SIG Testing]


Following metrics from kube-controller-manager are now marked as with the ALPHA stability level:
storage_count_attachable_volumes_in_use
attachdetach_controller_total_volumes
pv_collector_bound_pv_count
pv_collector_unbound_pv_count
pv_collector_bound_pvc_count
pv_collector_unbound_pvc_count (#84896, @RainbowMango) [SIG Apps]


Renamed FeatureGate RequestManagement to APIPriorityAndFairness.  This feature gate is an alpha and has not yet been associated with any actual functionality.
change references to feature gate RequestManagement into references to APIPriorityAndFairness (#85260, @MikeSpreitzer) [SIG API Machinery]


--runtime-config now supports an api/beta=false value which disables all built-in REST API versions matching v[0-9]+beta[0-9]+.
--feature-gates now supports an AllBeta=false value which disables all beta feature gates. (#84304, @liggitt) [SIG API Machinery, and SIG Cluster Lifecycle]


kube-proxy now supports DualStack feature with EndpointSlices and IPVS. (#85246, @robscott) [SIG Network]


Enables VolumeSnapshotDataSource feature gate and promotes volume snapshot APIs to beta. (#80058, @xing-yang) [SIG Cluster Lifecycle, SIG Storage, and SIG Testing]


kubeadm alpha certs command now skip missing files (#85092, @fabriziopandini) [SIG Cluster Lifecycle]


A new flag "progress-report-url" has been added to the test context which allows progress information about the test run to be sent to a webhook. In addition, this information is printed to stdout to aid in users watching the logs. (#84524, @johnSchnake) [SIG Testing]


Following metrics from kubelet are now marked as with the ALPHA stability level:
node_cpu_usage_seconds_total
node_memory_working_set_bytes
container_cpu_usage_seconds_total
container_memory_working_set_bytes
scrape_error (#84987, @RainbowMango) [SIG API Machinery, SIG Cluster Lifecycle, SIG Instrumentation, and SIG Node]


Following metrics from kubelet are now marked as with the ALPHA stability level:
kubelet_container_log_filesystem_used_bytes
kubelet_volume_stats_capacity_bytes
kubelet_volume_stats_available_bytes
kubelet_volume_stats_used_bytes
kubelet_volume_stats_inodes
kubelet_volume_stats_inodes_free
kubelet_volume_stats_inodes_used
plugin_manager_total_plugins
volume_manager_total_volumes (#84907, @RainbowMango) [SIG Node]


kubeadm: enable the usage of the secure kube-scheduler and kube-controller-manager ports for health checks. For kube-scheduler was 10251, becomes 10259. For kube-controller-manager was 10252, becomes 10257. (#85043, @neolit123) [SIG Cluster Lifecycle]


Mirror pods now include an ownerReference for the node that created them. (#84485, @tallclair) [SIG Node, and SIG Testing]


Bump CSI version to 1.2.0 (#84832, @gnufied) [SIG Storage]


Existing PVs are converted to use volume topology if migration is enabled. (#83394, @bertinatto) [SIG Apps, SIG Scheduling, and SIG Storage]


Deprecated metric kubeproxy_sync_proxy_rules_latency_microseconds has been turned off. (#83839, @RainbowMango) [SIG Network]


Finalizer Protection for Service LoadBalancers is now in GA (enabled by default). This feature ensures the Service resource is not fully deleted until the correlating load balancer resources are deleted. (#85023, @MrHohn) [SIG Apps, and SIG Network]


New flag --show-hidden-metrics-for-version in kube-apiserver can be used to show all hidden metrics that deprecated in the previous minor release. (#84292, @RainbowMango) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Instrumentation]


The ResourceQuotaScopeSelectors feature has graduated to GA. The ResourceQuotaScopeSelectors feature gate is now unconditionally enabled and will be removed in 1.18. (#82690, @draveness) [SIG Scheduling]


kubeadm: add a upgrade health check that deploys a Job (#81319, @neolit123) [SIG Cluster Lifecycle]


Promote CSIMigrationAWS to Beta (off by default since it requires installation of the AWS EBS CSI Driver)
The in-tree AWS EBS plugin "kubernetes.io/aws-ebs" is now deprecated and will be removed in 1.21. Users should enable CSIMigration + CSIMigrationAWS features and install the AWS EBS CSI Driver (https://github.com/kubernetes-sigs/aws-ebs-csi-driver) to avoid disruption to existing Pod and PVC objects at that time.
Users should start using the AWS EBS CSI CSI Driver directly for any new volumes. (#85237, @leakingtapan) [SIG Storage]


Following metrics have been turned off:

apiserver_request_count
apiserver_request_latencies
apiserver_request_latencies_summary
apiserver_dropped_requests
etcd_request_latencies_summary
apiserver_storage_transformation_latencies_microseconds
apiserver_storage_data_key_generation_latencies_microseconds
apiserver_storage_transformation_failures_total (#83837, @RainbowMango) [SIG API Machinery, and SIG Testing]


The official kube-proxy image (used by kubeadm, among other things) is now
compatible with systems running iptables 1.8 in "nft" mode, and will autodetect
which mode it should use. (#82966, @danwinship) [SIG Network]


Kubenet: added HostPort IPv6 support
HostPortManager: operates only with one IP family, failing if receives portmapping entries with different IP families
HostPortSyncer: operates only with one IP family, skipping portmap entries with different IP families (#80854, @aojea) [SIG Network, and SIG Node]


Feature gates CSIMigration to Beta (on by default) and CSIMigrationGCE to Beta (off by default since it requires installation of the GCE PD CSI Driver)
The in-tree GCE PD plugin "kubernetes.io/gce-pd" is now deprecated and will be removed in 1.21. Users should enable CSIMigration + CSIMigrationGCE features and install the GCE PD CSI Driver (https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver) to avoid disruption to existing Pod and PVC objects at that time.
Users should start using the GCE PD CSI CSI Driver directly for any new volumes. (#85231, @davidz627) [SIG Apps, SIG Auth, SIG Node, SIG Storage, and SIG Testing]


Following metrics have been turned off:

scheduler_scheduling_latency_seconds
scheduler_e2e_scheduling_latency_microseconds
scheduler_scheduling_algorithm_latency_microseconds
scheduler_scheduling_algorithm_predicate_evaluation
scheduler_scheduling_algorithm_priority_evaluation
scheduler_scheduling_algorithm_preemption_evaluation
scheduler_scheduling_binding_latency_microseconds (#83838, @RainbowMango) [SIG Scheduling]


kubeadm now supports automatic calculations of dual-stack node cidr masks to kube-controller-manager. (#85609, @Arvinderpal) [SIG Cluster Lifecycle]


kubeadm: reset raises warnings if it cannot delete folders (#85265, @SataQiu) [SIG Cluster Lifecycle]


kubelet now exports a "server_expiration_renew_failure" and "client_expiration_renew_failure" metric counter if the certificate rotations cannot be performed. (#84614, @rphillips) [SIG API Machinery, SIG Auth, SIG CLI, SIG Cloud Provider, SIG Cluster Lifecycle, SIG Instrumentation, SIG Node, and SIG Release]


Profiling is enabled by default in the scheduler (#84835, @denkensk) [SIG Scheduling]


Scheduler now reports metrics on cache size including nodes, pods, and assumed pods (#83508, @damemi) [SIG Instrumentation, and SIG Scheduling]


azure: Add allow unsafe read from cache (#83685, @aramase) [SIG Cloud Provider]


update the latest validated version of Docker to 19.03 (#84476, @neolit123) [SIG Cluster Lifecycle]


User can now use component config to configure NodeLabel plugin for the scheduler framework. (#84297, @liu-cong) [SIG Scheduling]


local: support local filesystem volume with block resource reconstruction (#84218, @cofyc) [SIG Node, SIG Storage, and SIG Testing]


kubelet: a configuration file specified via --config is now loaded with strict deserialization, which fails if the config file contains duplicate or unknown fields. This protects against accidentally running with config files that are malformed, mis-indented, or have typos in field names, and getting unexpected behavior. (#83204, @obitech) [SIG Cluster Lifecycle, and SIG Node]


kubeadm now propagates proxy environment variables to kube-proxy (#84559, @yastij) [SIG Cluster Lifecycle]


Reload apiserver SNI certificates from disk every minute (#84303, @jackkleeman) [SIG API Machinery, and SIG Testing]


Update Azure SDK versions to v35.0.0 (#84543, @andyzhangx) [SIG Cloud Provider]


Adding initial EndpointSlice metrics. (#83257, @robscott) [SIG Apps, and SIG Network]


add azure disk encryption(SSE+CMK) support (#84605, @andyzhangx) [SIG Cloud Provider, and SIG Storage]


Reduce default NodeStatusReportFrequency to 5 minutes. With this change, periodic node status updates will be send every 5m if node status doesn't change (otherwise they are still send with 10s).
Bump NodeProblemDetector version to v0.8.0 to reduce forced NodeStatus updates frequency to 5 minutes. (#84007, @wojtek-t) [SIG Cluster Lifecycle, SIG Node, SIG Scalability, and SIG Testing]


Added kubelet serving certificate metric server_rotation_seconds which is a histogram reporting the age of a just rotated serving certificate in seconds. (#84534, @sambdavidson) [SIG API Machinery, SIG Auth, SIG Instrumentation, and SIG Node]


Pod labels can no longer be updated through the pod/status updates by nodes. (#84260, @tallclair) [SIG Auth, and SIG Node]


Reload apiserver serving certificate from disk every minute (#84200, @jackkleeman) [SIG API Machinery, SIG Auth, SIG Node, and SIG Testing]


Add permit_wait_duration_seconds metric for scheduler. (#84011, @liu-cong) [SIG Scheduling]


Optimize inter-pod affinity preferredDuringSchedulingIgnoredDuringExecution type, up to 4x in some cases. (#84264, @ahg-g) [SIG Scheduling]


Refactor scheduler's framework permit API. (#83756, @hex108) [SIG Scheduling, and SIG Testing]


The kubectl's api-resource command now has a --sort-by flag to sort resources by name or kind. (#81971, @laddng) [SIG CLI]


Graduate TaintNodesByCondition to GA in 1.17. (feature gate will be removed in 1.18) action required (#82703, @draveness) [SIG API Machinery, SIG Apps, SIG Cloud Provider, SIG Node, SIG Scheduling, and SIG Testing]


Update to Ingress-GCE v1.6.1 (#84018, @rramkumar1) [SIG Cluster Lifecycle]


Graduate ScheduleDaemonSetPods to GA. (feature gate will be removed in 1.18) action required. (#82795, @draveness) [SIG Apps, SIG Scheduling, and SIG Testing]


Fixed binding of block PersistentVolumes / PersistentVolumeClaims when BlockVolume feature is off. (#84049, @jsafrane) [SIG Apps, and SIG Storage]


filter plugin for cloud provider storage predicate (#84148, @gongguan) [SIG Scheduling, and SIG Testing]


Expand scheduler priority functions and scheduling framework plugins' node score range to [0, 100]. action required. Note: this change is internal and does not affect extender and RequestedToCapacityRatio custom priority, which are still expected to provide a [0, 10] range. (#83522, @draveness) [SIG Scheduling]


Change pod_preemption_victims metric from Gauge to Histogram. (#83603, @Tabrizian) [SIG Scheduling]


Expose SharedInformerFactory in the framework handle (#83663, @draveness) [SIG Apps, SIG Scheduling, and SIG Testing]


The topology manager aligns resources for pods of all QoS classes with respect to NUMA locality, not just Guaranteed QoS pods. (#83492, @ConnorDoyle) [SIG Node]


kubeadm: enhance certs check-expiration to show the expiration info of related CAs (#83932, @SataQiu) [SIG Cluster Lifecycle]


Add incoming pods metrics to scheduler queue. (#83577, @liu-cong) [SIG Scheduling]


Allow dynamically set glog logging level of kube-scheduler (#83910, @mrkm4ntr) [SIG Scheduling]


Add latency and request count metrics for scheduler framework. (#83569, @liu-cong) [SIG Scheduling]


ETCD version monitor metrics are now marked as with the ALPHA stability level. (#83283, @RainbowMango) [SIG Cluster Lifecycle]


A new --prefix flag added into kubectl logs which prepends each log line with information about it's source (pod name and container name) (#76471, @m1kola) [SIG CLI]


Kubeadm: add support for 127.0.0.1 as advertise address. kubeadm will automatically replace this value with matching global unicast IP address on the loopback interface. (#83475, @fabriziopandini) [SIG API Machinery, and SIG Cluster Lifecycle]


kube-scheduler: a configuration file specified via --config is now loaded with strict deserialization, which fails if the config file contains duplicate or unknown fields. This protects against accidentally running with config files that are malformed, mis-indented, or have typos in field names, and getting unexpected behavior. (#83030, @obitech) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Scheduling]


kubeadm: use the --service-cluster-ip-range flag to init or use the ServiceSubnet field in the kubeadm config to pass a comma separated list of Service CIDRs. (#82473, @Arvinderpal) [SIG Cluster Lifecycle]


Bump version of event-exporter to 0.3.1, to switch it to protobuf. (#83396, @loburm) [SIG Instrumentation, and SIG Scalability]


Remove MaxPriority in the scheduler API, please use MaxNodeScore or MaxExtenderPriority instead. (#83386, @draveness) [SIG Scheduling, and SIG Testing]


Add per-pod scheduling metrics across 1 or more schedule attempts. (#83674, @liu-cong) [SIG Scheduling]


The mutating and validating admission webhook plugins now read configuration from the admissionregistration.k8s.io/v1 API. (#80883, @liggitt) [SIG API Machinery]


kube-proxy: a configuration file specified via --config is now loaded with strict deserialization, which fails if the config file contains duplicate or unknown fields. This protects against accidentally running with config files that are malformed, mis-indented, or have typos in field names, and getting unexpected behavior. (#82927, @obitech) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Network]


kubeadm: implemented structured output of 'kubeadm token list' in JSON, YAML, Go template and JsonPath formats (#78764, @bart0sh) [SIG Cluster Lifecycle]


Expose kubernetes client in the scheduling framework handle. (#82432, @draveness) [SIG Scheduling]


Added Clone method to the scheduling framework's PluginContext and ContextData. (#82951, @ahg-g) [SIG Scheduling]


Update crictl to v1.16.1. (#82856, @Random-Liu) [SIG Cluster Lifecycle, and SIG Node]


Reduces the number of calls made to the Azure API when requesting the instance view of a virtual machine scale set node. (#82496, @hasheddan) [SIG Cloud Provider]


Consolidate ScoreWithNormalizePlugin into the ScorePlugin interface (#83042, @draveness) [SIG Scheduling, and SIG Testing]


New APIs to allow adding/removing pods from pre-calculated prefilter state in the scheduling framework (#82912, @ahg-g) [SIG Scheduling, and SIG Testing]


Added metrics 'authentication_latency_seconds' that can be used to understand the latency of authentication. (#82409, @RainbowMango) [SIG API Machinery, SIG Auth, and SIG Instrumentation]


Modified the scheduling framework's Filter API. (#82842, @ahg-g) [SIG Scheduling, and SIG Testing]


Added cloud operation count metrics to azure cloud controller manager. (#82574, @kkmsft) [SIG Cloud Provider]


Design


kubeadm now errors out whenever a not supported component config version is supplied for the kubelet and kube-proxy (#85639, @rosti) [SIG Cluster Lifecycle]

Documentation


BREAKING CHANGE: Remove plugin watching of deprecated directory {kubelet_root_dir}/plugins and CSI V0 support in accordance with deprecation announcement in https://v1-13.docs.kubernetes.io/docs/setup/release/notes/ (#84533, @davidz627) [SIG Node, SIG Storage, and SIG Testing]
Updated kube-proxy ipvs README with correct grep argument to list loaded ipvs modules (#83677, @pete911) [SIG Network]

Failing-Test


CSI Migration: Fixes issue where all volumes with the same inline volume inner spec name were staged in the same path. Migrated inline volumes are now staged at a unique path per unique volume. (#84754, @davidz627) [SIG Storage]
CSI Migration: GCE PD access mode now reflects read only status of inline volumes - this allows multi-attach for read only many PDs (#84809, @davidz627) [SIG Storage]

Other (Bug, Cleanup or Flake)


kubectl drain node --dry-run will list pods that would be evicted or deleted (#82660, @sallyom) [SIG CLI]


EndpointSlice hostname is now set in the same conditions Endpoints hostname is. (#84207, @robscott) [SIG Apps, and SIG Network]


kube-apiserver: fixed a bug that could cause a goroutine leak if the apiserver encountered an encoding error serving a watch to a websocket watcher (#84693, @tedyu) [SIG API Machinery]


Simple script based hyperkube image that bundles all the necessary binaries. This is a equivalent replacement for the image based on the go based hyperkube command + image. (#84662, @dims) [SIG Cluster Lifecycle, and SIG Release]


configmaps/extension-apiserver-authentication in kube-system is continuously updated by kube-apiservers, instead of just at apiserver start (#82705, @deads2k) [SIG API Machinery, SIG Auth, and SIG Testing]


kubeadm: fix an issue with the kube-proxy container env. variables (#84888, @neolit123) [SIG Cluster Lifecycle]


Change GCP ILB firewall names to contain the "k8s-fw-" prefix like the rest of the firewall rules. This is needed for consistency and also for other components to identify the firewall rule as k8s/service-controller managed. (#84622, @prameshj) [SIG Cloud Provider]


The example API server has renamed its wardle.k8s.io API group to wardle.example.com (#81670, @liggitt) [SIG API Machinery, and SIG Testing]


Utilize diagnostics tool to dump GKE windows test logs (#83517, @YangLu1031) [SIG Cluster Lifecycle]


When the go-client reflector relists, the ResourceVersion list option is set to the reflector's latest synced resource version to ensure the reflector does not "go back in time" and reprocess events older than it has already processed. If the the server responds with an HTTP 410 (Gone) status code response, the relist falls back to using resourceVersion="". (#83520, @jpbetz) [SIG API Machinery, and SIG Node]


Improving the performance of Endpoint and EndpointSlice controllers by caching Service Selectors (#84280, @gongguan) [SIG Apps, and SIG Network]


Critical pods can now be created in namespaces other than kube-system. To limit critical pods to the kube-system namespace, cluster admins should create an admission configuration file limiting critical pods by default, and a matching quota object in the kube-system namespace permitting critical pods in that namespace. See https://kubernetes.io/docs/concepts/policy/resource-quotas/&#35;limit-priority-class-consumption-by-default for details. (#76310, @ravisantoshgudimetla) [SIG Scheduling, and SIG Testing]


kubeadm no longer defaults or validates the component configs of the kubelet or kube-proxy (#79223, @rosti) [SIG Cluster Lifecycle]


kubeadm: fix skipped etcd upgrade on secondary control-plane nodes when the command "kubeadm upgrade node" is used. (#85024, @neolit123) [SIG Cluster Lifecycle]


fix race condition when attach/delete azure disk in same time (#84917, @andyzhangx) [SIG Cloud Provider]


If given an IPv6 bind-address, kube-apiserver will now advertise an IPv6 endpoint for the kubernetes.default service. (#84727, @danwinship) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Network]


kube-apiserver: Fixed a regression accepting patch requests > 1MB (#84963, @liggitt) [SIG API Machinery, and SIG Testing]


update github.com/vishvananda/netlink to v1.0.0 (#83576, @andrewsykim) [SIG Network]


All nodes need to be drained before upgrading Kubernetes cluster, because paths used for block volumes are changed in this release, so on-line upgrade of nodes aren't allowed. (#74026, @mkimuram) [SIG Node, and SIG Storage]


kubectl: --resource-version now works properly in label/annotate/set selector commands when racing with other clients to update the target object (#85285, @liggitt) [SIG CLI, and SIG Testing]


Add table convertor to componentstatus. (#85174, @zhouya0) [SIG API Machinery, and SIG CLI]


kubeadm: added retry to all the calls to the etcd API so kubeadm will be more resilient to network glitches (#85201, @fabriziopandini) [SIG Cluster Lifecycle]


kubelet and aggregated API servers now use v1 TokenReview and SubjectAccessReview endpoints to check authentication/authorization.

kube-apiserver can now specify --authentication-token-webhook-version=v1 or --authorization-webhook-version=v1 to use v1 TokenReview and SubjectAccessReview API objects when communicating with authentication and authorization webhooks. (#84768, @liggitt) [SIG API Machinery, SIG Auth, SIG Node, and SIG Testing]


matches := stackCreator.FindStringSubmatch(string(stack))
matches type is []string
so nil len(matches) ==0
change:
if matches == nil || len(matches) != 4 ==>if len(matches) != 4 (#84957, @ZP-AlwaysWin) [SIG API Machinery, and SIG Scheduling]


Fix a racing issue in client-go UpdateTransportConfig. (#80284, @danielqsj) [SIG API Machinery, and SIG Auth]


kubeadm: remove the deprecated "--cri-socket" flag for "kubeadm upgrade apply". The flag has been deprecated since v1.14. (#85044, @neolit123) [SIG Cluster Lifecycle]


kubeadm deprecates the use of the hyperkube image (#85094, @rosti) [SIG Cluster Lifecycle]


Clients can request protobuf and json and correctly negotiate with the server for JSON for CRD objects, allowing all client libraries to request protobuf if it is available.  If an error occurs negotiating a watch with the server, the error is immediately return by the client Watch() method instead of being sent as an Error event on the watch stream. (#84692, @smarterclayton) [SIG API Machinery, SIG Auth, SIG CLI, SIG Network, and SIG Testing]


kubeadm: prevent potential hanging of commands such as "kubeadm reset" if the apiserver endpoint is not reachable. (#84648, @neolit123) [SIG Cluster Lifecycle]


azure: update disk lock logic per vm during attach/detach to allow concurrent updates for different nodes. (#85115, @aramase) [SIG Cloud Provider]


Scale custom resource unconditionally if resourceVersion is not provided (#80572, @knight42) [SIG API Machinery, and SIG CLI]


When using Containerd on Windows, the TerminationMessagePath file will now be mounted in the Windows Pod. (#83057, @claudiubelu) [SIG Node, and SIG Windows]


apiservers based on k8s.io/apiserver with delegated authn based on cluster authentication will automatically update to new authentication information when the authoritative configmap is updated. (#85004, @deads2k) [SIG API Machinery, SIG Auth, and SIG Testing]


fix vmss dirty cache issue in disk attach/detach on vmss node (#85158, @andyzhangx) [SIG Cloud Provider]


Kubeadm now includes CoreDNS version 1.6.5

kubernetes plugin adds metrics to measure kubernetes control plane latency.
the health plugin now includes the lameduck option by default, which waits for a duration before shutting down. (#85109, @rajansandeep) [SIG Cluster Lifecycle]


Fixes a bug in kubeadm that caused init and join to hang indefinitely in specific conditions. (#85156, @chuckha) [SIG Cluster Lifecycle]


kube-apiserver: Authentication configuration for mutating and validating admission webhooks referenced from an --admission-control-config-file can now be specified with apiVersion: apiserver.config.k8s.io/v1, kind: WebhookAdmissionConfiguration. (#85138, @liggitt) [SIG API Machinery]


kube-apiserver: The ResourceQuota admission plugin configuration referenced from --admission-control-config-file admission config has been promoted to apiVersion: apiserver.config.k8s.io/v1, kind: ResourceQuotaConfiguration with no schema changes. (#85099, @liggitt) [SIG API Machinery]


Fixed bug when using kubeadm alpha certs commands with clusters using external etcd (#85091, @fabriziopandini) [SIG Cluster Lifecycle]


Fix incorrect network policy description suggesting that pods are isolated when a network policy has no rules of a given type (#84194, @jackkleeman) [SIG CLI, and SIG Network]


Fix a bug that a node Lease object may have been created without OwnerReference. (#84998, @wojtek-t) [SIG Node, and SIG Testing]


kubeadm: add a new "kubelet-finalize" phase as part of the "init" workflow and an experimental sub-phase to enable automatic kubelet client certificate rotation on primary control-plane nodes.
Prior to 1.17 and for existing nodes created by "kubeadm init" where kubelet client certificate rotation is desired, you must modify "/etc/kubernetes/kubelet.conf" to point to the PEM symlink for rotation:
"client-certificate: /var/lib/kubelet/pki/kubelet-client-current.pem" and "client-key: /var/lib/kubelet/pki/kubelet-client-current.pem", replacing the embedded client certificate and key. (#84118, @neolit123) [SIG Cluster Lifecycle]


Filter published OpenAPI schema by making nullable, required fields non-required in order to avoid kubectl to wrongly reject null values. (#85722, @sttts) [SIG API Machinery]


kubectl set resources will no longer return an error if passed an empty change for a resource.
kubectl set subject will no longer return an error if passed an empty change for a resource. (#85490, @sallyom) [SIG CLI]


kube-apiserver: fixed a conflict error encountered attempting to delete a pod with gracePeriodSeconds=0 and a resourceVersion precondition (#85516, @michaelgugino) [SIG API Machinery]


CRDs can have fields named type with value array and nested array with items fields without validation to fall over this. (#85223, @sttts) [SIG API Machinery, SIG CLI, SIG Cloud Provider, SIG Cluster Lifecycle, SIG Instrumentation, and SIG Node]


Resolves error from v1.17.0-beta.2 with --authorizer-mode webhook complaining about an invalid version (#85441, @liggitt) [SIG API Machinery, and SIG Auth]


EndpointSlices are not enabled by default. Use the EndpointSlice feature gate to enable this feature. (#85365, @robscott) [SIG Auth, and SIG Network]


kubeadm: Fix a bug where kubeadm cannot parse kubelet's version if the latter dumps logs on the standard error. (#85351, @rosti) [SIG Cluster Lifecycle]


When upgrading to 1.17 with a cluster with EndpointSlices enabled, the endpointslice.kubernetes.io/managed-by label needs to be set on each EndpointSlice. (#85359, @robscott) [SIG Apps, SIG Network, and SIG Testing]


kube-controller-manager: Fixes bug setting headless service labels on endpoints (#85361, @liggitt) [SIG Apps, and SIG Network]


Remove redundant API validation when using Service Topology with externalTrafficPolicy=Local (#85346, @andrewsykim) [SIG Network]


kubeadm: make sure images are pre-pulled even if a tag did not change but their contents changed (#85603, @bart0sh) [SIG Cluster Lifecycle]


kube-apiserver: Fixes a bug that hidden metrics can not be enabled by the command-line option --show-hidden-metrics-for-version. (#85444, @RainbowMango) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Instrumentation]


Fix bug where EndpointSlice controller would attempt to modify shared objects. (#85368, @robscott) [SIG API Machinery, SIG Apps, and SIG Network]


Use context to check client closed instead of http.CloseNotifier in processing watch request which will reduce 1 goroutine for each request if proto is HTTP/2.x . (#85408, @answer1991) [SIG API Machinery]


Wait for kubelet & kube-proxy to be ready on Windows node within 10s (#85228, @YangLu1031) [SIG Cluster Lifecycle]


kubeadm: fix a panic in case the KubeProxyConfiguration feature gates were not initialized. (#85524, @Arvinderpal) [SIG Cluster Lifecycle]


kubeadm: fix stray "node-cidr-mask-size" flag in the kube-controller-manager manifest when IPv6DualStack is enabled (#85494, @tedyu) [SIG Cluster Lifecycle]


Fixed issue with addon-resizer using deprecated extensions APIs (#85793, @bskiba) [SIG Cluster Lifecycle, and SIG Instrumentation]


Includes FSType when describing CSI persistent volumes. (#85293, @huffmanca) [SIG CLI, and SIG Storage]


kubeadm: don't write the kubelet environment file on "upgrade apply" (#85412, @boluisa) [SIG Cluster Lifecycle]


fix azure file AuthorizationFailure (#85475, @andyzhangx) [SIG Cloud Provider, and SIG Storage]


Resolved regression in admission, authentication, and authorization webhook performance in v1.17.0-rc.1 (#85810, @liggitt) [SIG API Machinery, and SIG Testing]


kubeadm: uses the apiserver AdvertiseAddress IP family to choose the etcd endpoint IP family for non external etcd clusters (#85745, @aojea) [SIG Cluster Lifecycle]


kubeadm: Forward cluster name to the controller-manager arguments (#85817, @ereslibre) [SIG Cluster Lifecycle]


Fixed "requested device X but found Y" attach error on AWS. (#85675, @jsafrane) [SIG Cloud Provider, and SIG Storage]


Update Cluster Autoscaler to 1.17.0; changelog: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.17.0 (#85610, @losipiuk) [SIG Autoscaling, and SIG Cluster Lifecycle]


kubeadm: retry kubeadm-config ConfigMap creation or mutation if the apiserver is not responding. This will improve resiliency when joining new control plane nodes. (#85763, @ereslibre) [SIG Cluster Lifecycle]


addons: elasticsearch discovery supports IPv6 (#85543, @SataQiu) [SIG Cluster Lifecycle, and SIG Instrumentation]


Kubernetes now requires go1.13.4+ to build (#82809, @liggitt) [SIG API Machinery, SIG Auth, SIG CLI, SIG Cloud Provider, SIG Cluster Lifecycle, SIG Instrumentation, SIG Network, SIG Node, SIG Release, SIG Storage, and SIG Testing]


Ensure health probes are created for local traffic policy UDP services on Azure (#84802, @feiskyer) [SIG Cloud Provider]


kube-proxy: emits a warning when a malformed component config file is used with v1alpha1. (#84143, @phenixblue) [SIG Network]


Update default etcd server version to 3.4.3 (#84329, @jingyih) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Testing]


Scheduler policy configs can no longer be declared multiple times (#83963, @damemi) [SIG Scheduling]


This PR sets the --cluster-dns flag value to kube-dns service IP whether or not NodeLocal DNSCache is enabled. NodeLocal DNSCache will listen on both the link-local as well as the service IP. (#84383, @prameshj) [SIG Cluster Lifecycle, and SIG Network]


Remove prometheus cluster monitoring addon from kube-up (#83442, @serathius) [SIG Cluster Lifecycle, and SIG Testing]


kubeadm: always mount the kube-controller-manager hostPath volume that is given by the --flex-volume-plugin-dir flag. (#84468, @neolit123) [SIG Cluster Lifecycle]


kube-scheduler now fallbacks to emitting events using core/v1 Events when events.k8s.io/v1beta1 is disabled. (#83692, @yastij) [SIG API Machinery, SIG Apps, SIG Scheduling, and SIG Testing]


local: support local volume block mode reconstruction (#84173, @cofyc) [SIG Node, SIG Storage, and SIG Testing]


Fixed kubectl endpointslice output for get requests (#82603, @robscott) [SIG CLI]


set config.BindAddress to IPv4 address "127.0.0.1" if not specified (#83822, @zouyee) [SIG Network]


CSI detach timeout increased from 10 seconds to 2 minutes (#84321, @cduchesne) [SIG Storage]


deprecate cleanup-ipvs flag (#83832, @gongguan) [SIG Network]


Fixed a bug in the single-numa-policy of the TopologyManager. Previously, best-effort pods would result in a terminated state with a TopologyAffinity error. Now they will run as expected. (#83777, @lmdaly) [SIG Node]


Fix the bug that EndpointSlice for masters wasn't created after enabling EndpointSlice feature on a pre-existing cluster. (#84421, @tnqn) [SIG API Machinery]


sourcesReady provides the readiness of kubelet configuration sources such as apiserver update readiness. (#81344, @zouyee) [SIG Cluster Lifecycle, and SIG Node]


kube-scheduler: emits a warning when a malformed component config file is used with v1alpha1. (#84129, @obitech) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Scheduling]


The certificate signer no longer accepts ca.key passwords via the CFSSL_CA_PK_PASSWORD environment variable. This capability was not prompted by user request, never advertised, and recommended against in the security audit. (#84677, @mikedanese) [SIG API Machinery, SIG Apps, SIG Auth, SIG CLI, and SIG Node]


Only validate duplication of the RequestedToCapacityRatio custom priority and allow other custom predicates/priorities (#84646, @liu-cong) [SIG Scheduling]


Removed Alpha feature MountContainers (#84365, @codenrhoden) [SIG Node, and SIG Storage]


People can see the right log and note. (#84637, @zhipengzuo) [SIG Apps, and SIG Network]


The built-in system:csi-external-provisioner and system:csi-external-attacher cluster roles are removed as of 1.17 release (#84282, @tedyu) [SIG Auth, and SIG Storage]


Add data cache flushing during unmount device for GCE-PD driver in Windows Server. (#83591, @jingxu97) [SIG Storage, and SIG Windows]


Adds a metric apiserver_request_error_total to kube-apiserver. This metric tallies the number of request_errors encountered by verb, group, version, resource, subresource, scope, component, and code. (#83427, @logicalhan) [SIG API Machinery, and SIG Instrumentation]


None. (#84138, @nilo19) [SIG Cloud Provider]


Update to use go1.12.12 (#84064, @cblecker) [SIG Release, and SIG Testing]


Update Cluster Autoscaler version to 1.16.2 (CA release docs: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.16.2) (#84038, @losipiuk) [SIG Cluster Lifecycle]


client-ca bundles for the all generic-apiserver based servers will dynamically reload from disk on content changes (#83579, @deads2k) [SIG API Machinery, SIG Auth, SIG Node, and SIG Testing]


Reduced frequency of DescribeVolumes calls of AWS API when attaching/detaching a volume. (#84181, @jsafrane) [SIG Cloud Provider, and SIG Storage]


Add a metric to track number of scheduler binding and prioritizing goroutines (#83535, @wgliang) [SIG Scheduling]


Fix kubelet metrics gathering on non-English Windows hosts (#84156, @wawa0210) [SIG Node, and SIG Windows]


A new kubelet_preemptions metric is reported from Kubelets to track the number of preemptions occuring over time, and which resource is triggering those preemptions. (#84120, @smarterclayton) [SIG Instrumentation, SIG Node, and SIG Scheduling]


TaintNodesByCondition was graduated to GA, CheckNodeMemoryPressure, CheckNodePIDPressure, CheckNodeDiskPressure, CheckNodeCondition were accidentally removed since 1.12, the replacement is to use CheckNodeUnschedulablePred (#84152, @draveness) [SIG Scheduling]


[migration phase 1] PodFitsResources as framework plugin (#83650, @wgliang) [SIG Scheduling]


Fixed attachment of AWS volumes that have just been detached. (#83567, @jsafrane) [SIG Cloud Provider, and SIG Storage]


[migration phase 1] PodMatchNodeSelector/NodAffinity as filter plugin (#83660, @wgliang) [SIG Scheduling]


Upgrade to etcd client 3.3.17 to fix bug where etcd client does not parse IPv6 addresses correctly when members are joining, and to fix bug where failover on multi-member etcd cluster fails certificate check on DNS mismatch (#83801, @jpbetz) [SIG API Machinery, and SIG Cloud Provider]


Fixed panic when accessing CustomResources of a CRD with x-kubernetes-int-or-string. (#83787, @sttts) [SIG API Machinery]


Add more tracing steps in generic_scheduler (#83539, @wgliang) [SIG Scheduling]


[migration phase 1] PodFitsHost as filter plugin (#83662, @wgliang) [SIG Scheduling]


Fix unsafe JSON construction in a number of locations in the codebase (#81158, @zouyee) [SIG API Machinery, SIG Apps, and SIG Node]


kubeadm no longer removes /etc/cni/net.d as it does not install it. Users should remove files from it manually or rely on the component that created them (#83950, @yastij) [SIG Cluster Lifecycle]


Switched intstr.Type to sized integer to follow API guidelines and improve compatibility with proto libraries (#83956, @liggitt) [SIG API Machinery]


Fix handling tombstones in pod-disruption-budged controller. (#83951, @zouyee) [SIG Apps]


client-go: improved allocation behavior of the delaying workqueue when handling objects with far-future ready times. (#83945, @barkbay) [SIG API Machinery]


Added the crictl Windows binaries as well as the Linux 32bit binary to the release archives (#83944, @saschagrunert) [SIG Release]


Fixed an issue with informers missing an Added event if a recently deleted object was immediately recreated at the same time the informer dropped a watch and relisted. (#83911, @matte21) [SIG API Machinery]


clean duplicate GetPodServiceMemberships function (#83902, @gongguan) [SIG Apps, and SIG Network]


Bumps metrics-server version to v0.3.6 with following bugfix:

Don't break metric storage when duplicate pod metrics encountered causing hpa to fail (#83907, @olagacek) [SIG Cluster Lifecycle, and SIG Instrumentation]


Gives the right error message when using kubectl delete a wrong resource. (#83825, @zhouya0) [SIG CLI]


The userspace mode of kube-proxy no longer confusingly logs messages about deleting endpoints that it is actually adding. (#83644, @danwinship) [SIG Network]


Significant kube-proxy performance improvements when using Endpoint Slices at scale. (#83206, @robscott) [SIG Apps, and SIG Network]


Ceph RBD volume plugin now does not use any keyring (/etc/ceph/ceph.client.lvs01cinder.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin) for authentication. Ceph user credentials must be provided in PersistentVolume objects and referred Secrets. (#75588, @smileusd) [SIG Storage]


Upgrade default etcd server version to 3.3.17 (#83804, @jpbetz) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Testing]


[migration phase 1] PodFitsHostPorts as filter plugin (#83659, @wgliang) [SIG Scheduling]


more complete and accurate logging of stack backtraces in E2E failures (#82176, @pohly) [SIG API Machinery, SIG CLI, SIG Cloud Provider, SIG Cluster Lifecycle, and SIG Testing]


Rename PluginContext to CycleState in the scheduling framework (#83430, @draveness) [SIG Scheduling, and SIG Testing]


Significant kube-proxy performance improvements for non UDP ports. (#83208, @robscott) [SIG Network]


The resource version option, when passed to a list call, is now consistently interpreted as the minimum allowed resource version.  Previously when listing resources that had the watch cache disabled clients could retrieve a snapshot at that exact resource version.  If the client requests a resource version newer than the current state, a TimeoutError is returned suggesting the client retry in a few seconds.  This behavior is now consistent for both single item retrieval and list calls, and for when the watch cache is enabled or disabled. (#72170, @jpbetz) [SIG API Machinery]


Fixes a flaw (CVE-2019-11253) in json/yaml decoding where large or malformed documents could consume excessive server resources. Request bodies for normal API requests (create/delete/update/patch operations of regular resources) are now limited to 3MB. (#83261, @liggitt) [SIG API Machinery, SIG CLI, SIG Cloud Provider, SIG Cluster Lifecycle, and SIG Testing]


Some scheduler extender API fields are moved from pkg/scheduler/api to pkg/scheduler/apis/extender/v1. (#83262, @Huang-Wei) [SIG Scheduling, and SIG Testing]


Fixes a goroutine leak in kube-apiserver when a request times out. (#83333, @lavalamp) [SIG API Machinery]


Fixed a bug in the single-numa-node policy of the TopologyManager.  Previously, pods that only requested CPU resources and did not request any third-party devices would fail to launch with a TopologyAffinity error. Now they will launch successfully. (#83697, @klueska) [SIG Node]


Fix validation message to mention bytes, not characters. (#80880, @DirectXMan12) [SIG API Machinery, and SIG Testing]


Fix error where metrics related to dynamic kubelet config isn't registered (#83184, @odinuge) [SIG Node]


Openstack: Do not delete managed LB in case of security group reconciliation errors (#82264, @multi-io) [SIG Cloud Provider]


kubeadm: when adding extra apiserver authorization-modes, the defaults "Node,RBAC" are no longer prepended in the resulting static Pod manifests and a full override is allowed. (#82616, @ghouscht) [SIG Cluster Lifecycle]


Authentication token cache size is increased (from 4k to 32k) to support clusters with many nodes or many namespaces with active service accounts. (#83643, @lavalamp) [SIG API Machinery, and SIG Auth]


Bumps the minimum version of Go required for building Kubernetes to 1.12.4. (#83596, @jktomer) [SIG Release]


kube-proxy iptables probabilities are now more granular and will result in better distribution beyond 319 endpoints. (#83599, @robscott) [SIG Network]


If container fails because ContainerCannotRun, do not utilize the FallbackToLogsOnError TerminationMessagePolicy, as it masks more useful logs. (#81280, @yqwang-ms) [SIG Node]


If a bad flag is supplied to a kubectl command, only a tip to run --help is printed, instead of the usage menu.  Usage menu is printed upon running kubectl command --help. (#82423, @sallyom) [SIG CLI]


Fixed the bug that deleted services were processed by EndpointSliceController repeatedly even their cleanup were successful. (#82996, @tnqn) [SIG Apps]


Fixed cleanup of raw block devices after kubelet restart. (#83451, @jsafrane) [SIG Node, SIG Storage, and SIG Testing]


Commands like kubectl apply now return errors if schema-invalid annotations are specified, rather than silently dropping the entire annotations section. (#83552, @liggitt) [SIG CLI]


kubeadm.k8s.io/v1beta1 has been deprecated, you should update your config to use newer non-deprecated API versions. (#83276, @Klaven) [SIG Cluster Lifecycle]


kubeadm: fix wrong default value for the "upgrade node --certificate-renewal" flag. (#83528, @neolit123) [SIG Cluster Lifecycle]


IP validates if a string is a valid IP address (#83104, @zouyee) [SIG Storage]


The --certificate-authority flag now correctly overrides existing skip TLS or CA data settings in the kubeconfig file (#83547, @liggitt) [SIG API Machinery, and SIG CLI]


hyperkube will now be available in a new github repository and will not be included in the kubernetes release from 1.17 onwards (#83454, @dims) [SIG Cluster Lifecycle, and SIG Release]


Use ipv4 in wincat port forward. (#83036, @liyanhui1228) [SIG Node]


Bump metrics-server to v0.3.5 (#83015, @olagacek) [SIG Cluster Lifecycle]


dashboard: disable the dashboard Deployment on non-Linux nodes. This step is required to support Windows worker nodes. (#82975, @wawa0210) [SIG Cluster Lifecycle]


Fix possible fd leak and closing of dirs when using openstack (#82873, @odinuge) [SIG Cloud Provider, and SIG Storage]


PersistentVolumeLabel admission plugin, responsible for labeling PersistentVolumes with topology labels, now does not overwrite existing labels on PVs that were dynamically provisioned. It trusts the  dynamic provisioning that it provided the correct labels to the PersistentVolume, saving one potentially expensive cloud API call. PersistentVolumes created manually by users are labelled by the admission plugin in the same way as before. (#82830, @jsafrane) [SIG Storage]


Fix aggressive VM calls for Azure VMSS (#83102, @feiskyer) [SIG Cloud Provider]


Update Azure load balancer to prevent orphaned public IP addresses (#82890, @chewong) [SIG Apps, SIG Cloud Provider, and SIG Network]


Use online nodes instead of possible nodes when discovering available NUMA nodes (#83196, @zouyee) [SIG Node]


Fixes the bug in informer-gen that it produces incorrect code if a type has nonNamespaced tag set. (#80458, @tatsuhiro-t) [SIG API Machinery]


Update to go 1.12.10 (#83139, @cblecker) [SIG Release, and SIG Testing]


On AWS nodes with multiple network interfaces, kubelet should now more reliably report the same primary node IP. (#80747, @danwinship) [SIG Cloud Provider, SIG Network, and SIG Node]


Fixes kube-proxy bug accessing self nodeip:port on windows (#83027, @liggitt) [SIG Network, and SIG Windows]


Resolves bottleneck in internal API server communication that can cause increased goroutines and degrade API Server performance (#80465, @answer1991) [SIG API Machinery]


The deprecated mondo kubernetes-test tarball is no longer built. Users running Kubernetes e2e tests should use the kubernetes-test-portable and kubernetes-test-{OS}-{ARCH} tarballs instead. (#83093, @ixdy) [SIG Release, and SIG Testing]


Improved performance of kube-proxy with EndpointSlice enabled with more efficient sorting. (#83035, @robscott) [SIG Network]


# kubectl rollout history sts/test-sts
statefulset.apps/test-sts
REVISION
0
0
0
1
2
3 (#82643, @ZP-AlwaysWin) [SIG CLI]


Conformance tests may now include disruptive tests. If you are running tests against a live cluster, consider skipping those tests tagged as Disruptive to avoid non-test workloads being impacted. Be aware, skipping any conformance tests (even disruptive ones) will make the results ineligible for consideration for the CNCF Certified Kubernetes program. (#82664, @johnSchnake) [SIG Architecture, and SIG Testing]


Resolves regression generating informers for packages whose names contain . characters (#82410, @nikhita) [SIG API Machinery]


kube-dns add-on:

All containers are now being executed under more restrictive privileges.
Most of the containers now run as non-root user and has the root filesystem set as read-only.
The remaining container running as root only has the minimum Linux capabilities it requires to run.
Privilege escalation has been disabled for all containers. (#82347, @pjbgf) [SIG Cluster Lifecycle, and SIG Network]


k8s dockerconfigjson secrets are now compatible with docker config desktop authentication credentials files (#82148, @bbourbie) [SIG Auth, and SIG Node]


The docker container runtime now enforces a 220 second timeout on container network operations. (#71653, @liucimin) [SIG Network, and SIG Node]


Fixed a scheduler panic when using PodAffinity. (#82841, @Huang-Wei) [SIG Scheduling]


Fixes a panic in kube-controller-manager cleaning up bootstrap tokens (#82887, @tedyu) [SIG Cluster Lifecycle]


Fix panic in kubelet when running IPv4/IPv6 dual-stack mode with a CNI plugin (#82508, @aanm) [SIG Network, and SIG Node]


Kubernetes no longer monitors firewalld. On systems using firewalld for firewall
maintenance, kube-proxy will take slightly longer to recover from disruptive
firewalld operations that delete kube-proxy's iptables rules.
As a side effect of these changes, kube-proxy's
sync_proxy_rules_last_timestamp_seconds metric no longer behaves the
way it used to; now it will only change when services or endpoints actually
change, rather than reliably updating every 60 seconds (or whatever). If you
are trying to monitor for whether iptables updates are failing, the
sync_proxy_rules_iptables_restore_failures_total metric may be more useful. (#81517, @danwinship) [SIG Cluster Lifecycle, SIG Network, SIG Node, and SIG Testing]


The deprecated feature gates GCERegionalPersistentDisk, EnableAggregatedDiscoveryTimeout and PersistentLocalVolumes are now unconditionally enabled and can no longer be specified in component invocations. (#82472, @draveness) [SIG API Machinery, SIG Storage, and SIG Testing]


Report non-confusing error for negative storage size in PVC spec. (#82759, @sttts) [SIG Apps, and SIG Storage]


Resolves issue with /readyz and /livez not including etcd and kms health checks (#82713, @logicalhan) [SIG API Machinery]


fix: azure disk detach failure if node not exists (#82640, @andyzhangx) [SIG Cloud Provider]