KEP-3895: Interactive(-i) flag to kubectl delete for user confirmation

Implementation History
STABLE Implemented
Created 2023-03-01
Latest v1.30
Milestones
Alpha v1.28
Beta v1.29
Stable v1.30
Related Enhancements
Ownership
Owning SIG
SIG CLI
Primary Authors
@ardaguclu

KEP-3895: Interactive(-i) flag to kubectl delete for user confirmation

Release Signoff Checklist

Items marked with (R) are required prior to targeting to a milestone / release.

  • (R) Enhancement issue in release milestone, which links to KEP dir in kubernetes/enhancements (not the initial KEP PR)
  • (R) KEP approvers have approved the KEP status as implementable
  • (R) Design details are appropriately documented
  • (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
    • e2e Tests for all Beta API Operations (endpoints)
    • (R) Ensure GA e2e tests meet requirements for Conformance Tests
    • (R) Minimum Two Week Window for GA e2e tests to prove flake free
  • (R) Graduation criteria is in place
  • (R) Production readiness review completed
  • (R) Production readiness review approved
  • “Implementation History” section is up-to-date for milestone
  • User-facing documentation has been created in kubernetes/website , for publication to kubernetes.io
  • Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes

Summary

Introduce interactive mode for kubectl delete command, to allow users protect cluster administrators from irrevocable removal of critical resources.

Motivation

kubectl delete is disruptive and irreversible command and should be used cautiously. However, there are various circumstances such as inaccurate usage, mistyping, hasty tab completions, etc. that this command can position the cluster to a state where it is not intended to be. Due to the backwards compatibility concerns, seeking confirmation from a user before proceeding to genuinely delete as default, is out of the table.

On the other hand, we need to provide a way to mitigate accidental deletes to users and this KEP proposes a new flag, namely interactive(-i) that will ask confirmation from a user to proceed with deletion.

Goals

  • Introduce interactive(-i) flag to be used for asking confirmation to user to proceed actual delete operation or not.

Non-Goals

  • Smart solutions on client side to detect deletion is accidental or not
  • Deletion protection mechanism on server side

Proposal

When a user passes --interactive/-i flag to kubectl delete command, all the resources planned to be deleted will be shown to the user as preview. Command continues deletion process if user confirms by typing y. If user types n or any other value other than y, process will be cancelled and command returns a message with zero exit code.

$ kubectl delete --interactive -f test_deployment.yaml 
You are about to delete the following 2 resource(s):
Deployment/dockerd
Deployment/dockerd2
Do you want to continue? (y/n): n
deletion is cancelled

$ kubectl delete -i -f test_deployment.yaml
You are about to delete the following 2 resource(s):
Deployment/dockerd
Deployment/dockerd2
Do you want to continue? (y/n): y
deployment.apps "dockerd" deleted
deployment.apps "dockerd2" deleted

For the alpha phase, this flag will be hidden behind KUBECTL_INTERACTIVE_DELETE environment variable and can only be use when this KUBECTL_INTERACTIVE_DELETE=true environment variable is exported.

Implementation Details/Design

One of major advantage of this feature is the ability to preview all resources before deletion. That means we need to traverse all resources twice. First one is for display, and the other one is for deletion. However, that comes along with a technical limitation that Visit function does not support traversing twice as stated in here . But we also cannot use Infos function as suggested in the link above, because that unties the builder configuration and deletion logic.

In order to overcome this limitation, this feature will use separate builder to show resources as preview. Current builder will continue being used as deletion traverse, other builder will be used for preview. Builders must match in terms of configurations to prevent such discrepancies and required tests will be added to assure that configurations are same or not. Moreover, deletion will only be performed to the resources whose uidMap match the ones in the preview list.

User Stories (Optional)

As a user, I want to have an option via a flag to list all the resources being deleted as preview and proceed to actual deletion only after explicit confirmation.

Story 1

As filed in this issue https://github.com/kubernetes/kubectl/issues/1365; user tries to delete all persistentvolumeclaims in a namespace;

$ kubectl delete --all pvc -n yournamespace

However, mistyping pv instead of pvc deletes all persistentvolumes in the cluster, since persistentvolumes are non-namespaced resources.

Story 2

As filed in this issue https://github.com/kubernetes/kubectl/issues/696; user executes kubectl delete --all namespaces without enough knowledge about the consequences of this command.

Notes/Constraints/Caveats (Optional)

  • In terms of backwards compatibility, this flag’s default will always be false.
  • If user cancels deletion, exit code will be 0.
  • This flag will not be used with --raw flag.
  • Set of filtered objects may be changed and user might need to execute multiple subsequent delete commands. Likeliness of this issue especially increases with user waits long between previewing and the confirming.

Risks and Mitigations

Only standard input via terminal will be supported because main motivation of this flag is to provide a way to users. That would be a risk if there is exceptional inputs whose are available for users that we can not handle.

Design Details

Test Plan

[x] I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement.

Prerequisite testing updates
Unit tests
  • k8s.io/kubectl/pkg/cmd/delete: 2023-03-01 - 76.7
Integration tests
  • new integration tests covering the interactive(-i) flag usage combination with other flags will be added.
e2e tests
  • e2e test will be added deleting a resource by simulating a fake input(“yes”) and expecting resource is actually deleted.

Graduation Criteria

Alpha

  • Feature implemented behind a new KUBECTL_INTERACTIVE_DELETE environment variable
  • Initial unit and integration tests are implemented

Beta

  • Gather feedback from developers and surveys
  • KUBECTL_INTERACTIVE_DELETE environment variable is removed and interactive flag is available by default

GA

  • e2e tests are implemented

Upgrade / Downgrade Strategy

N/A

Version Skew Strategy

N/A

Production Readiness Review Questionnaire

Feature Enablement and Rollback

How can this feature be enabled / disabled in a live cluster?
  • Other
    • Describe the mechanism: exporting KUBECTL_INTERACTIVE_DELETE=true environment variable will enable the feature.
    • Will enabling / disabling the feature require downtime of the control plane? No
    • Will enabling / disabling the feature require downtime or reprovisioning of a node? (Do not assume Dynamic Kubelet Config feature is enabled). No
Does enabling the feature change any default behavior?

No

Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?

Yes

What happens if we reenable the feature if it was previously rolled back?

User will continue deleting resources without being asked confirmation as now.

Are there any tests for feature enablement/disablement?

No, users can only use this feature via enabling environment variable. In addition to that, there will be a test case to assure that users will not be able to use this feature without setting environment variable.

Rollout, Upgrade and Rollback Planning

How can a rollout or rollback fail? Can it impact already running workloads?

No

What specific metrics should inform a rollback?

N/A

Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?

N/A

Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.?

N/A

Monitoring Requirements

N/A

How can an operator determine if the feature is in use by workloads?

If user is being asked for confirmation, when running delete command.

How can someone using this feature know that it is working for their instance?
  • Other (treat as last resort)
    • Details: If user is being asked for confirmation, when running delete command.
What are the reasonable SLOs (Service Level Objectives) for the enhancement?

N/A

What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service?

N/A

Are there any missing metrics that would be useful to have to improve observability of this feature?

N/A

Dependencies

Does this feature depend on any specific services running in the cluster?

No

Scalability

Will enabling / using this feature result in any new API calls?

No

Will enabling / using this feature result in introducing new API types?

No

Will enabling / using this feature result in any new calls to the cloud provider?

No

Will enabling / using this feature result in increasing size or count of the existing API objects?

No

Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?

No

Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, …) in any components?

No

Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)?

No

Troubleshooting

How does this feature react if the API server and/or etcd is unavailable?

Interactive flag will not be able to get resources and return error just like how default kubectl delete command reacts.

What are other known failure modes?

N/A

What steps should be taken if SLOs are not being met to determine the problem?

Implementation History

The KEP was proposed on 2023-03-01 The KEP was promoted to beta on 2023-10-02 The KEP was promoted to stable on 2024-01-09

Drawbacks

N/A

Alternatives

Confirming Per Resource (yes/no)

$ kubectl delete pod -l group=abc --interactive
Are you sure you want to delete pod/bar? (y/n): y
pod "bar" deleted
Are you sure you want to delete pod/baz? (y/n): y
pod "baz" deleted
Are you sure you want to delete pod/foo? (y/n): y
pod "foo" deleted
  • Pros:
    • Supporting partial deletion
    • No need to use separate builder, we can ask confirmation via traversing resources.
  • Cons:
    • Not showing resources as preview
    • Might be tedious if there are multiple resources

Confirming Per Resource with Yes to All Option(yes/no/yes to all)

$ kubectl delete pod -l group=abc --interactive
Remove pod/bar? (A/y/n): y
pod "bar" deleted
Remove pod/baz? (A/y/n): A
pod "baz" deleted
pod "foo" deleted

If user does not pass --all or --selector, it is very likely that user tries to delete a few resources, so we will ask yes or no per resource. Otherwise, if user passes --all or --selector, there will be additional yes to all(A) option to confirm the rest.

  • Pros:
    • Intuitive for Unix users
    • Supporting partial deletion
    • No need to use separate builder, we can ask confirmation via traversing resources.
  • Cons:
    • Not showing resources as preview. Yes to all means that “you can delete the rest, even though I don’t know”