KEP-5495: Deprecate ipvs mode in kube-proxy
KEP-5495: Deprecate IPVS mode in kube-proxy
- Release Signoff Checklist
- Summary
- Motivation
- Proposal
- Design Details
- Production Readiness Review Questionnaire
- Implementation History
- Drawbacks
- Alternatives
- Infrastructure Needed (Optional)
Release Signoff Checklist
Items marked with (R) are required prior to targeting to a milestone / release.
- (R) Enhancement issue in release milestone, which links to KEP dir in kubernetes/enhancements (not the initial KEP PR)
- (R) KEP approvers have approved the KEP status as
implementable - (R) Design details are appropriately documented
- (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
- e2e Tests for all Beta API Operations (endpoints)
- (R) Ensure GA e2e tests meet requirements for Conformance Tests
- (R) Minimum Two Week Window for GA e2e tests to prove flake free
- (R) Graduation criteria is in place
- (R) all GA Endpoints must be hit by Conformance Tests
- (R) Production readiness review completed
- (R) Production readiness review approved
- “Implementation History” section is up-to-date for milestone
- User-facing documentation has been created in kubernetes/website , for publication to kubernetes.io
- Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
Summary
This KEP proposes deprecation of ipvs in kube-proxy.
Motivation
At time of writing, kube-proxy has four supported backends (iptables, ipvs, nftables, and winkernel).
sig-network currently lacks maintainers who are familar with the ipvs backend code, and as such, has been encouraging
users who report ipvs bugs to move to using the nftables backend mode, where they can. (ie: first example
, second example
)
ipvs was introduced in KEP-265
to solve performance problems in large clusters.
KEP-3866
was created to introduce a new nftables mode to kube-proxy. The goal of this new backend mode
has always been to eventually replace ipvs and iptables1, as it solve the performance issues of iptables
and already solves many of the bugs present in the ipvs mode.
Goals
- Deprecate the
ipvsmode of kube-proxy
Non-Goals
- Removal of the
ipvsmode of kube-proxy - The specifics of removal ofipvsare being handled in KEP-5343
Proposal
Notes/Constraints/Caveats (Optional)
The specifics of removal of ipvs mode will be handled in KEP-5343
Risks and Mitigations
N/A
Design Details
Test Plan
[X] I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement.
Prerequisite testing updates
Unit tests
N/A - This KEP is only adding a deprecation warning and updating necessary parts of the website
Integration tests
N/A - This KEP is only adding a deprecation warning and updating nessesary parts of the website
e2e tests
N/A - This KEP is only adding a deprecation warning and updating nessesary parts of the website
Graduation Criteria
Deprecation
- The Kubernetes web site has been updated with deprecation notices for the
ipvsmode of kube-proxy - Kube-proxy prints a warning when a user starts kube-proxy in
ipvsmode - All nftables-mode bugfixes have been backported to 1.34 and 1.33, to ensure that
ipvsusers on older releases can still migrate tonftables.
Upgrade / Downgrade Strategy
N/A - The only code change is a warning notice
Version Skew Strategy
N/A - The only code change is a warning notice
Production Readiness Review Questionnaire
Feature Enablement and Rollback
How can this feature be enabled / disabled in a live cluster?
N/A
Does enabling the feature change any default behavior?
N/A
Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
N/A
What happens if we reenable the feature if it was previously rolled back?
Are there any tests for feature enablement/disablement?
N/A
Rollout, Upgrade and Rollback Planning
N/A
How can a rollout or rollback fail? Can it impact already running workloads?
N/A
What specific metrics should inform a rollback?
N/A
Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?
N/A
Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.?
N/A
Monitoring Requirements
N/A
How can an operator determine if the feature is in use by workloads?
N/A
How can someone using this feature know that it is working for their instance?
N/A
What are the reasonable SLOs (Service Level Objectives) for the enhancement?
N/A
What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service?
N/A
Are there any missing metrics that would be useful to have to improve observability of this feature?
N/A
Dependencies
Does this feature depend on any specific services running in the cluster?
N/A
Scalability
N/A
Will enabling / using this feature result in any new API calls?
N/A
Will enabling / using this feature result in introducing new API types?
N/A
Will enabling / using this feature result in any new calls to the cloud provider?
N/A
Will enabling / using this feature result in increasing size or count of the existing API objects?
N/A
Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?
N/A
Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, …) in any components?
N/A
Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)?
N/A
Troubleshooting
How does this feature react if the API server and/or etcd is unavailable?
What are other known failure modes?
N/A
What steps should be taken if SLOs are not being met to determine the problem?
Implementation History
N/A
Drawbacks
Users may be running version of the kernel which isn’t new enough to support the nftables backend mode in kube-proxy.
Alternatives
Getting active maintainers for ipvs may be a short term alternative, see The ipvs mode of kube-proxy will not save us
(from KEP-3866
) for details
Infrastructure Needed (Optional)
N/A