Skip to content

Instantly share code, notes, and snippets.

@saschagrunert

saschagrunert/v1.17.0-beta.0-v1.17.0-beta.1.md

Created November 26, 2019 10:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save saschagrunert/a7c13d8b3e6a45d5c0f0c8f9da1e004b to your computer and use it in GitHub Desktop.
Save saschagrunert/a7c13d8b3e6a45d5c0f0c8f9da1e004b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
v1.17.0-beta.0-v1.17.0-beta.1.md

New Features

  • Following metrics have been turned off:

    • apiserver_request_count
    • apiserver_request_latencies
    • apiserver_request_latencies_summary
    • apiserver_dropped_requests
    • etcd_request_latencies_summary
    • apiserver_storage_transformation_latencies_microseconds
    • apiserver_storage_data_key_generation_latencies_microseconds
    • apiserver_storage_transformation_failures_total (#83837, @RainbowMango) [SIG API Machinery, and SIG Testing]

  • The official kube-proxy image (used by kubeadm, among other things) is now compatible with systems running iptables 1.8 in "nft" mode, and will autodetect which mode it should use. (#82966, @danwinship) [SIG Network]

  • Kubenet: added HostPort IPv6 support HostPortManager: operates only with one IP family, failing if receives portmapping entries with different IP families HostPortSyncer: operates only with one IP family, skipping portmap entries with different IP families (#80854, @aojea) [SIG Network, and SIG Node]

  • Feature gates CSIMigration to Beta (on by default) and CSIMigrationGCE to Beta (off by default since it requires installation of the GCE PD CSI Driver) The in-tree GCE PD plugin "kubernetes.io/gce-pd" is now deprecated and will be removed in 1.21. Users should enable CSIMigration + CSIMigrationGCE features and install the GCE PD CSI Driver (https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver) to avoid disruption to existing Pod and PVC objects at that time. Users should start using the GCE PD CSI CSI Driver directly for any new volumes. (#85231, @davidz627) [SIG Apps, SIG Auth, SIG Node, SIG Storage, and SIG Testing]

  • Following metrics have been turned off:

    • scheduler_scheduling_latency_seconds
    • scheduler_e2e_scheduling_latency_microseconds
    • scheduler_scheduling_algorithm_latency_microseconds
    • scheduler_scheduling_algorithm_predicate_evaluation
    • scheduler_scheduling_algorithm_priority_evaluation
    • scheduler_scheduling_algorithm_preemption_evaluation
    • scheduler_scheduling_binding_latency_microseconds (#83838, @RainbowMango) [SIG Scheduling]

  • Support Service Topology (#72046, @m1093782566) [SIG Apps, SIG Auth, and SIG Network]

  • kube-controller-manager --node-cidr-mask-size-ipv4 int32 Default: 24. Mask size for IPv4 node-cidr in dual-stack cluster. --node-cidr-mask-size-ipv6 int32 Default: 64. Mask size for IPv6 node-cidr in dual-stack cluster.

    These 2 flags can be used only for dual-stack clusters. For non dual-stack clusters, continue to use --node-cidr-mask-size flag to configure the mask size.

    The default node cidr mask size for IPv6 was 24 which is now changed to 64. (#79993, @aramase) [SIG API Machinery, SIG Apps, SIG Cloud Provider, SIG Network, and SIG Testing]

  • --runtime-config now supports an api/beta=false value which disables all built-in REST API versions matching v[0-9]+beta[0-9]+. --feature-gates now supports an AllBeta=false value which disables all beta feature gates. (#84304, @liggitt) [SIG API Machinery, and SIG Cluster Lifecycle]

  • kube-proxy now supports DualStack feature with EndpointSlices and IPVS. (#85246, @robscott) [SIG Network]

  • Bump CSI version to 1.2.0 (#84832, @gnufied) [SIG Storage]

  • Adds Windows Server build information as a label on the node. (#84472, @gab-satchi) [SIG Node, and SIG Windows]

  • Deprecated metric kubeproxy_sync_proxy_rules_latency_microseconds has been turned off. (#83839, @RainbowMango) [SIG Network]

  • Existing PVs are converted to use volume topology if migration is enabled. (#83394, @bertinatto) [SIG Apps, SIG Scheduling, and SIG Storage]

  • Finalizer Protection for Service LoadBalancers is now in GA (enabled by default). This feature ensures the Service resource is not fully deleted until the correlating load balancer resources are deleted. (#85023, @MrHohn) [SIG Apps, and SIG Network]

  • EndpointSlices are now beta and enabled by default for better Network Endpoint performance at scale. (#84390, @robscott) [SIG API Machinery, SIG Apps, SIG Auth, SIG CLI, SIG Network, and SIG Testing]

  • New flag --show-hidden-metrics-for-version in kube-apiserver can be used to show all hidden metrics that deprecated in the previous minor release. (#84292, @RainbowMango) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Instrumentation]

  • The ResourceQuotaScopeSelectors feature has graduated to GA. The ResourceQuotaScopeSelectors feature gate is now unconditionally enabled and will be removed in 1.18. (#82690, @draveness) [SIG Scheduling]

  • Pod process namespace sharing is now Generally Available. The PodShareProcessNamespace feature gate is now deprecated and will be removed in Kubernetes 1.19. (#84356, @verb) [SIG Apps, SIG Node, and SIG Testing]

  • add RequiresExactMatch for label.Selector (#85048, @shaloulcy) [SIG API Machinery]

  • Deprecated metric rest_client_request_latency_seconds has been turned off. (#83836, @RainbowMango) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Instrumentation]

  • Removed dependency on kubectl from several storage E2E tests (#84042, @okartau) [SIG Storage, and SIG Testing]

  • Add plugin_execution_duration_seconds metric for scheduler framework plugins. (#84522, @liu-cong) [SIG Scheduling]

  • Node-specific volume limits has graduated to GA. (#83568, @bertinatto) [SIG Auth, SIG Node, SIG Scheduling, SIG Storage, and SIG Testing]

  • Adds a new label to indicate what is managing an EndpointSlice. (#83965, @robscott) [SIG Apps, and SIG Network]

  • Enables VolumeSnapshotDataSource feature gate and promotes volume snapshot APIs to beta. (#80058, @xing-yang) [SIG Cluster Lifecycle, SIG Storage, and SIG Testing]

  • kubeadm alpha certs command now skip missing files (#85092, @fabriziopandini) [SIG Cluster Lifecycle]

  • A new flag "progress-report-url" has been added to the test context which allows progress information about the test run to be sent to a webhook. In addition, this information is printed to stdout to aid in users watching the logs. (#84524, @johnSchnake) [SIG Testing]

  • Following metrics from kubelet are now marked as with the ALPHA stability level: node_cpu_usage_seconds_total node_memory_working_set_bytes container_cpu_usage_seconds_total container_memory_working_set_bytes scrape_error (#84987, @RainbowMango) [SIG API Machinery, SIG Cluster Lifecycle, SIG Instrumentation, and SIG Node]

  • Following metrics from kubelet are now marked as with the ALPHA stability level: kubelet_container_log_filesystem_used_bytes kubelet_volume_stats_capacity_bytes kubelet_volume_stats_available_bytes kubelet_volume_stats_used_bytes kubelet_volume_stats_inodes kubelet_volume_stats_inodes_free kubelet_volume_stats_inodes_used plugin_manager_total_plugins volume_manager_total_volumes (#84907, @RainbowMango) [SIG Node]

  • kubeadm: enable the usage of the secure kube-scheduler and kube-controller-manager ports for health checks. For kube-scheduler was 10251, becomes 10259. For kube-controller-manager was 10252, becomes 10257. (#85043, @neolit123) [SIG Cluster Lifecycle]

  • Mirror pods now include an ownerReference for the node that created them. (#84485, @tallclair) [SIG Node, and SIG Testing]

  • kubeadm: the command "kubeadm token create" now has a "--certificate-key" flag that can be used for the formation of join commands for control-planes with automatic copy of certificates (#84591, @TheLastProject) [SIG Cluster Lifecycle]

  • Promote NodeLease feature to GA. The feature make Lease object changes an additional healthiness signal from Node. Together with that, we reduce frequency of NodeStatus updates to 5m by default in case of no changes to status itself (#84351, @wojtek-t) [SIG API Machinery, SIG Apps, SIG Auth, SIG Node, and SIG Testing]

  • Following metrics from kube-controller-manager are now marked as with the ALPHA stability level: storage_count_attachable_volumes_in_use attachdetach_controller_total_volumes pv_collector_bound_pv_count pv_collector_unbound_pv_count pv_collector_bound_pvc_count pv_collector_unbound_pvc_count (#84896, @RainbowMango) [SIG Apps]

  • Updated EndpointSlices to use PublishNotReadyAddresses from Services. (#84573, @robscott) [SIG Apps, and SIG Network]

  • A new kubelet command line option, --reserved-cpus, is introduced to explicitly define the the CPU list that will be reserved for system. For example, if --reserved-cpus=0,1,2,3 is specified, then cpu 0,1,2,3 will be reserved for the system. On a system with 24 CPUs, the user may specify isolcpus=4-23 for the kernel option and use CPU 4-23 for the user containers. (#83592, @jianzzha) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Node]

  • Profiling is enabled by default in the scheduler (#84835, @denkensk) [SIG Scheduling]

  • Adding initial EndpointSlice metrics. (#83257, @robscott) [SIG Apps, and SIG Network]

Changes by kind

api-change

  • OpenAPI v3 format in CustomResourceDefinition schemas are now documented. (#85381, @sttts) [SIG API Machinery, and SIG Testing]

  • Implement the documented API semantics of list-type and map-type atomic to reject non-atomic sub-types. (#84722, @sttts) [SIG API Machinery]

  • kube-apiserver: The AdmissionConfiguration type accepted by --admission-control-config-file has been promoted to apiserver.config.k8s.io/v1 with no schema changes. (#85098, @liggitt) [SIG API Machinery]

  • Splitting IP address type into IPv4 and IPv6 for EndpointSlices (#84971, @robscott) [SIG Apps, SIG CLI, SIG Network, SIG Scheduling, and SIG Testing]

  • Moving WindowsRunAsUserName feature to beta (#84882, @marosset) [SIG Apps, SIG Node, and SIG Windows]

  • Added appProtocol field to EndpointSlice Port (#83815, @howardjohn) [SIG Apps, SIG Cluster Lifecycle, SIG Instrumentation, and SIG Network]

  • Deprecate the instance type beta label ("beta.kubernetes.io/instance-type") in favor of it's GA equivalent: "node.kubernetes.io/instance-type" (#82049, @andrewsykim) [SIG Apps, SIG Auth, SIG Cloud Provider, SIG Node, and SIG Scheduling]

  • Deprecate the beta labels for zones ("failure-domain.beta.kubernetes.io/zone") and regions ("failure-domain.beta.kubernetes.io/region") in favor of their GA equivalents: "topology.kubernetes.io/zone" and "topology.kubernetes.io/region".

    The beta labels "failure-domain.beta.kubernetes.io/zone" and "failure-domain.beta.kubernetes.io/region" will be removed in v1.21 (#81431, @andrewsykim) [SIG Apps, SIG Auth, SIG Cloud Provider, SIG Network, SIG Node, SIG Scheduling, SIG Storage, and SIG Testing]

  • CRDs defaulting is promoted to GA. Note: the feature gate CustomResourceDefaulting will be removed in 1.18. (#84713, @sttts) [SIG API Machinery, and SIG Testing]

  • All resources within the rbac.authorization.k8s.io/v1alpha1 and rbac.authorization.k8s.io/v1beta1 API groups are deprecated in favor of rbac.authorization.k8s.io/v1, and will no longer be served in v1.20. (#84758, @liggitt) [SIG Auth]

  • Scheduler ComponentConfig fields are now pointers (#83619, @damemi) [SIG Scheduling, and SIG Testing]

bug

  • Implement the documented API semantics of list-type and map-type atomic to reject non-atomic sub-types. (#84722, @sttts) [SIG API Machinery]
  • kubeadm: Fix a bug where kubeadm cannot parse kubelet's version if the latter dumps logs on the standard error. (#85351, @rosti) [SIG Cluster Lifecycle]
  • EndpointSlices are not enabled by default. Use the EndpointSlice feature gate to enable this feature. (#85365, @robscott) [SIG Auth, and SIG Network]
  • kube-controller-manager: Fixes bug setting headless service labels on endpoints (#85361, @liggitt) [SIG Apps, and SIG Network]
  • When upgrading to 1.17 with a cluster with EndpointSlices enabled, the endpointslice.kubernetes.io/managed-by label needs to be set on each EndpointSlice. (#85359, @robscott) [SIG Apps, SIG Network, and SIG Testing]
  • Remove redundant API validation when using Service Topology with externalTrafficPolicy=Local (#85346, @andrewsykim) [SIG Network]
  • CSI Migration: Fixes issue where all volumes with the same inline volume inner spec name were staged in the same path. Migrated inline volumes are now staged at a unique path per unique volume. (#84754, @davidz627) [SIG Storage]
  • kubectl: --resource-version now works properly in label/annotate/set selector commands when racing with other clients to update the target object (#85285, @liggitt) [SIG CLI, and SIG Testing]
  • Add table convertor to componentstatus. (#85174, @zhouya0) [SIG API Machinery, and SIG CLI]
  • kubeadm: added retry to all the calls to the etcd API so kubeadm will be more resilient to network glitches (#85201, @fabriziopandini) [SIG Cluster Lifecycle]
  • azure: update disk lock logic per vm during attach/detach to allow concurrent updates for different nodes. (#85115, @aramase) [SIG Cloud Provider]
  • Scale custom resource unconditionally if resourceVersion is not provided (#80572, @knight42) [SIG API Machinery, and SIG CLI]
  • When using Containerd on Windows, the TerminationMessagePath file will now be mounted in the Windows Pod. (#83057, @bclau) [SIG Node, and SIG Windows]
  • apiservers based on k8s.io/apiserver with delegated authn based on cluster authentication will automatically update to new authentication information when the authoritative configmap is updated. (#85004, @deads2k) [SIG API Machinery, SIG Auth, and SIG Testing]
  • fix vmss dirty cache issue in disk attach/detach on vmss node (#85158, @andyzhangx) [SIG Cloud Provider]
  • Fixes a bug in kubeadm that caused init and join to hang indefinitely in specific conditions. (#85156, @chuckha) [SIG Cluster Lifecycle]
  • Fixed bug when using kubeadm alpha certs commands with clusters using external etcd (#85091, @fabriziopandini) [SIG Cluster Lifecycle]
  • Fix a bug that a node Lease object may have been created without OwnerReference. (#84998, @wojtek-t) [SIG Node, and SIG Testing]
  • Fix incorrect network policy description suggesting that pods are isolated when a network policy has no rules of a given type (#84194, @jackkleeman) [SIG CLI, and SIG Network]
  • Critical pods can now be created in namespaces other than kube-system. To limit critical pods to the kube-system namespace, cluster admins should create an admission configuration file limiting critical pods by default, and a matching quota object in the kube-system namespace permitting critical pods in that namespace. See https://kubernetes.io/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default for details. (#76310, @ravisantoshgudimetla) [SIG Scheduling, and SIG Testing]
  • Fix a racing issue in client-go UpdateTransportConfig. (#80284, @danielqsj) [SIG API Machinery, and SIG Auth]
  • Clients can request protobuf and json and correctly negotiate with the server for JSON for CRD objects, allowing all client libraries to request protobuf if it is available. If an error occurs negotiating a watch with the server, the error is immediately return by the client Watch() method instead of being sent as an Error event on the watch stream. (#84692, @smarterclayton) [SIG API Machinery, SIG Auth, SIG CLI, SIG Network, and SIG Testing]
  • kubeadm: prevent potential hanging of commands such as "kubeadm reset" if the apiserver endpoint is not reachable. (#84648, @neolit123) [SIG Cluster Lifecycle]
  • kubeadm: fix skipped etcd upgrade on secondary control-plane nodes when the command "kubeadm upgrade node" is used. (#85024, @neolit123) [SIG Cluster Lifecycle]
  • fix race condition when attach/delete azure disk in same time (#84917, @andyzhangx) [SIG Cloud Provider]
  • If given an IPv6 bind-address, kube-apiserver will now advertise an IPv6 endpoint for the kubernetes.default service. (#84727, @danwinship) [SIG API Machinery, SIG Cluster Lifecycle, and SIG Network]
  • kube-apiserver: Fixed a regression accepting patch requests > 1MB (#84963, @liggitt) [SIG API Machinery, and SIG Testing]
  • EndpointSlice hostname is now set in the same conditions Endpoints hostname is. (#84207, @robscott) [SIG Apps, and SIG Network]
  • configmaps/extension-apiserver-authentication in kube-system is continuously updated by kube-apiservers, instead of just at apiserver start (#82705, @deads2k) [SIG API Machinery, SIG Auth, and SIG Testing]
  • kubeadm: fix an issue with the kube-proxy container env. variables (#84888, @neolit123) [SIG Cluster Lifecycle]
  • Change GCP ILB firewall names to contain the "k8s-fw-" prefix like the rest of the firewall rules. This is needed for consistency and also for other components to identify the firewall rule as k8s/service-controller managed. (#84622, @prameshj) [SIG Cloud Provider]
  • Improving the performance of Endpoint and EndpointSlice controllers by caching Service Selectors (#84280, @gongguan) [SIG Apps, and SIG Network]
  • When the go-client reflector relists, the ResourceVersion list option is set to the reflector's latest synced resource version to ensure the reflector does not "go back in time" and reprocess events older than it has already processed. If the the server responds with an HTTP 410 (Gone) status code response, the relist falls back to using resourceVersion="". (#83520, @jpbetz) [SIG API Machinery, and SIG Node]
  • Ensure health probes are created for local traffic policy UDP services on Azure (#84802, @feiskyer) [SIG Cloud Provider]
  • CSI Migration: GCE PD access mode now reflects read only status of inline volumes - this allows multi-attach for read only many PDs (#84809, @davidz627) [SIG Storage]

cleanup

  • Remove redundant API validation when using Service Topology with externalTrafficPolicy=Local (#85346, @andrewsykim) [SIG Network]

  • update github.com/vishvananda/netlink to v1.0.0 (#83576, @andrewsykim) [SIG Network]

  • All nodes need to be drained before upgrading Kubernetes cluster, because paths used for block volumes are changed in this release, so on-line upgrade of nodes aren't allowed. (#74026, @mkimuram) [SIG Node, and SIG Storage]

  • kube-apiserver: Authentication configuration for mutating and validating admission webhooks referenced from an --admission-control-config-file can now be specified with apiVersion: apiserver.config.k8s.io/v1, kind: WebhookAdmissionConfiguration. (#85138, @liggitt) [SIG API Machinery]

  • Kubeadm now includes CoreDNS version 1.6.5

    • kubernetes plugin adds metrics to measure kubernetes control plane latency.
    • the health plugin now includes the lameduck option by default, which waits for a duration before shutting down. (#85109, @rajansandeep) [SIG Cluster Lifecycle]

  • kube-apiserver: The ResourceQuota admission plugin configuration referenced from --admission-control-config-file admission config has been promoted to apiVersion: apiserver.config.k8s.io/v1, kind: ResourceQuotaConfiguration with no schema changes. (#85099, @liggitt) [SIG API Machinery]

  • kube-apiserver: The AdmissionConfiguration type accepted by --admission-control-config-file has been promoted to apiserver.config.k8s.io/v1 with no schema changes. (#85098, @liggitt) [SIG API Machinery]

  • kubeadm no longer defaults or validates the component configs of the kubelet or kube-proxy (#79223, @rosti) [SIG Cluster Lifecycle]

  • matches := stackCreator.FindStringSubmatch(string(stack)) matches type is []string

    so nil len(matches) ==0

    change: if matches == nil || len(matches) != 4 ==>if len(matches) != 4 (#84957, @ZP-AlwaysWin) [SIG API Machinery, and SIG Scheduling]

  • kubelet and aggregated API servers now use v1 TokenReview and SubjectAccessReview endpoints to check authentication/authorization.

    • kube-apiserver can now specify --authentication-token-webhook-version=v1 or --authorization-webhook-version=v1 to use v1 TokenReview and SubjectAccessReview API objects when communicating with authentication and authorization webhooks. (#84768, @liggitt) [SIG API Machinery, SIG Auth, SIG Node, and SIG Testing]

  • BREAKING CHANGE: Remove plugin watching of deprecated directory {kubelet_root_dir}/plugins and CSI V0 support in accordance with deprecation announcement in https://v1-13.docs.kubernetes.io/docs/setup/release/notes/ (#84533, @davidz627) [SIG Node, SIG Storage, and SIG Testing]

  • kubeadm: remove the deprecated "--cri-socket" flag for "kubeadm upgrade apply". The flag has been deprecated since v1.14. (#85044, @neolit123) [SIG Cluster Lifecycle]

  • kube-apiserver: fixed a bug that could cause a goroutine leak if the apiserver encountered an encoding error serving a watch to a websocket watcher (#84693, @tedyu) [SIG API Machinery]

  • Simple script based hyperkube image that bundles all the necessary binaries. This is a equivalent replacement for the image based on the go based hyperkube command + image. (#84662, @dims) [SIG Cluster Lifecycle, and SIG Release]

  • The example API server has renamed its wardle.k8s.io API group to wardle.example.com (#81670, @liggitt) [SIG API Machinery, and SIG Testing]

  • Utilize diagnostics tool to dump GKE windows test logs (#83517, @YangLu1031) [SIG Cluster Lifecycle]

  • Kubernetes now requires go1.13.4+ to build (#82809, @liggitt) [SIG API Machinery, SIG Auth, SIG CLI, SIG Cloud Provider, SIG Cluster Lifecycle, SIG Instrumentation, SIG Network, SIG Node, SIG Release, SIG Storage, and SIG Testing]

  • All resources within the rbac.authorization.k8s.io/v1alpha1 and rbac.authorization.k8s.io/v1beta1 API groups are deprecated in favor of rbac.authorization.k8s.io/v1, and will no longer be served in v1.20. (#84758, @liggitt) [SIG Auth]

documentation

failing-test

  • CSI Migration: Fixes issue where all volumes with the same inline volume inner spec name were staged in the same path. Migrated inline volumes are now staged at a unique path per unique volume. (#84754, @davidz627) [SIG Storage]
  • CSI Migration: GCE PD access mode now reflects read only status of inline volumes - this allows multi-attach for read only many PDs (#84809, @davidz627) [SIG Storage]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment