KEP-2590: Kubectl Subresource Support

Implementation History
STABLE Implemented
Created 2021-04-06
Latest v1.33
Milestones
Alpha v1.24
Beta v1.27
Stable v1.33
Related Enhancements
Ownership
Owning SIG
SIG CLI
Participating SIGs
Arch
Primary Authors
@nikhita @ykakarap

KEP-2590: Kubectl Subresource Support

Release Signoff Checklist

Items marked with (R) are required prior to targeting to a milestone / release.

  • (R) Enhancement issue in release milestone, which links to KEP dir in kubernetes/enhancements (not the initial KEP PR)
  • (R) KEP approvers have approved the KEP status as implementable
  • (R) Design details are appropriately documented
  • (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
    • e2e Tests for all Beta API Operations (endpoints)
    • (R) Ensure GA e2e tests meet requirements for Conformance Tests
    • (R) Minimum Two Week Window for GA e2e tests to prove flake free
  • (R) Graduation criteria is in place
  • (R) Production readiness review completed
  • (R) Production readiness review approved
  • “Implementation History” section is up-to-date for milestone
  • User-facing documentation has been created in kubernetes/website , for publication to kubernetes.io
  • Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes

Summary

This document proposes adding a new --subresource flag to the following kubectl subcommands: get, patch, edit, apply and replace. The goal of this flag is to simplify the process of fetching and updating status, scale and resize subresources.

Motivation

Today while testing or debugging, fetching subresources (like status) of API objects via kubectl involves using kubectl --raw. Patching subresources using kubectl is not possible at all and requires using curl directly. This method is very cumbersome and not user-friendly.

This enhancement adds subresources as a first class option in kubectl to allow users to work with the API in a generic fashion.

Goals

  • Add a new flag --subresource=[subresource-name] to get, patch, edit, apply and replace kubectl commands to allow fetching and updating status, scale and resize subresources for all resources (built-in and custom resources) that support these.
  • Display pretty printed table columns for the status (uses same columns as the main resource) and scale subresources.

Non-Goals

  • Support subresources other than status and scale.
  • Allow specifying additionalPrinterColumns for CRDs for the status subresource.

Proposal

kubectl commands like get, patch, edit, apply and replace will now contain a new flag --subresource=[subresource-name] which will allow fetching and updating status, scale and resize subresources for all API resources.

Note that the API contract against the subresource is identical to a full resource. Therefore updating the status subresource to hold new value which could potentially be reconciled by a controller to a different value is expected behavior.

If --subresource flag is used for a resource that doesn’t support the subresource, a NotFound error will be returned.

User Stories (Optional)

Story 1

# for built-in types
# a `get` on a status subresource will return identical information
# to that of a full resource
$ kubectl get deployment nginx-deployment --subresource=status
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3/3     3            3           43s

$ kubectl get deployment nginx-deployment --subresource=scale
NAME               DESIREDREPLICAS   AVAILABLEREPLICAS
nginx-deployment   3                 3

# for CRDS
# a `get` on a status subresource will return identical information
# to that of a full resource
$ kubectl get crontab cron --subresource=status
NAME   SPEC          REPLICAS   AGE
cron   * * * * */5   3          4m52s

$ kubectl get crontab cron --subresource=scale
NAME   DESIREDREPLICAS   AVAILABLEREPLICAS
cron   3                 0

Invalid subresources:

$ kubectl get pod nginx-deployment-66b6c48dd5-dv6gl --subresource=logs
error: --subresource must be one of [status scale], not "logs"

$ kubectl get pod nginx-deployment-66b6c48dd5-dv6gl --subresource=scale
Error from server (NotFound): the server could not find the requested resource

Story 2

# For built-in types
# update spec.replicas through scale subresource
$ kubectl patch deployment nginx-deployment --subresource='scale' --type='merge' -p '{"spec":{"replicas":2}}'
scale.autoscaling/nginx-deployment patched

# spec.replicas is updated for the main resource
$ kubectl get deploy nginx-deployment
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   2/2     2            2           4m

# For CRDs
$ kubectl patch crontabs cron --subresource='status' --type='merge' -p '{"status":{"replicas":2}}'
crontab.stable.example.com/cron patched

$ kubectl get crontab cron --subresource=scale
NAME   DESIREDREPLICAS   AVAILABLEREPLICAS
cron   3                 2

Risks and Mitigations

This feature adds a new flag which will be validated like any other flag for a limited set of inputs. The remaining flags passed to every command will be validated as before.

Design Details

Subresource support

A new field Subresource and method Subresource/WithSubresource is added to the builder , helper and visitor code to use the API object at the subresource path.

For each relevant kubectl command, a new flag --subresource is added. If the flag is specified, the value is also validated to ensure that it is a valid subresource.

If the subresource does not exist for an API resource, a NotFound error is returned.

Table printer

To support table view for subresources using kubectl get, table convertor support is added to the scale and status subresources for built-in and custom resource types.

For built-in types, StatusStore and ScaleStore are updated to implement the TableConvertor interface. StatusStore uses the same columns as the main resource object.

The following column definitions for the Scale object are added to printers.go to support the scale subresource:

  • Available Replicas uses the json path .status.replicas of autoscalingv1.Scale object
  • Desired Replicas uses the json path of .spec.replicas of autoscalingv1.Scale object

For custom resources:

  • the status subresource uses the same columns as defined for the full resource, i.e., additionalPrinterColumns defined in the CRD.
  • the scale subresource follows the same column definitions as the built-in types, and are defined in helpers.go .

Test Plan

[x] I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement.

Unit tests
  • k8s.io/kubernetes/pkg/printers/internalversion: 2025-01-23 - 73.5
  • k8s.io/cli-runtime/pkg/resource: 2025-01-23 - 71.8
  • k8s.io/kubectl/pkg/cmd/apply: 2025-01-23 - 82
  • k8s.io/kubectl/pkg/cmd/edit: 2025-01-23 - 100
  • k8s.io/kubectl/pkg/cmd/get: 2025-01-23 - 80.8
  • k8s.io/kubectl/pkg/cmd/patch: 2025-01-23 - 56.4
  • k8s.io/kubectl/pkg/cmd/replace: 2025-01-23 - 63.8
Integration tests
e2e tests

Graduation Criteria

Alpha

  • Add the --subresource flag to get, patch, edit and replace subcommands.
  • Unit tests and integration tests are added.

Beta

  • Gather feedback from users.
  • e2e tests are added.
  • Add the --subresource flag to apply subcommand.

GA

Since v1.27 when the feature moved to beta, there have been no reported bugs concerning this feature. In fact, it is reassuring to see the community use this feature quite commonly such as in bug reports: https://github.com/kubernetes/kubernetes/issues/116311

Seeing this and given our added unit, integration and e2e tests gives us the confidence to graduate to stable.

Upgrade / Downgrade Strategy

See Version Skew Strategy .

Version Skew Strategy

The kube-apiserver functionality required for the --subresource flag to work correctly was introduced in Kubernetes v1.24. The current release (v1.33) exceeds the supported version skew policy . Therefore, there are no requirements for planning the upgrade or downgrade process. needs to be completed.

Production Readiness Review Questionnaire

Feature Enablement and Rollback

How can this feature be enabled / disabled in a live cluster?
  • Feature gate (also fill in values in kep.yaml)
    • Feature gate name:
    • Components depending on the feature gate:
  • Other
    • Describe the mechanism: A new flag for kubectl commands. For the alpha stage, description will be added to explicitly call out this flag as an alpha feature.
    • Will enabling / disabling the feature require downtime of the control plane? No, disabling the feature would be a client behavior.
    • Will enabling / disabling the feature require downtime or reprovisioning of a node? No, disabling the feature would be a client behavior.
Does enabling the feature change any default behavior?

While the feature now updates kubectl’s behavior to allow updating subresources, it is gated by the --subresource flag so it does not change kubectl’s default behavior.

Subresources can also be updated using curl today so this feature only provides a consistent way to use the API via kubectl, but does not allow additional changes to the API that are not possible today.

Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?

Yes, rolling back to a previous version of kubectl will remove support for the --subresource flag and the ability to update subresources via kubectl.

However, this does not “lock” us to any changes to the subresources that were made when the feature was enabled i.e. it does not remove the ability to update subresources for existing API resources completely. If a subresource of an existing API resource needs to be updated, this can be done via curl.

What happens if we reenable the feature if it was previously rolled back?

The --subresource flag can be used and subresources can be updated via kubectl again.

Are there any tests for feature enablement/disablement?

No, because it cannot be disabled or enabled in a single release.

Rollout, Upgrade and Rollback Planning

How can a rollout fail? Can it impact already running workloads?

The feature is encapsulated entirely within the kubectl binary, so rollout is an atomic client binary update. Subresources can always be updated via curl, so there are no version dependencies.

For kube-apiserver changes see Version Skew Strategy .

What specific metrics should inform a rollback?

N/A

Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?

This feature is completely within the client. The upgrades and rollback of cluster will not be affected by this change. The update and downgrade of the kubectl version will only limit the availability of the --subresource flag and will not change any API behavior.

For kube-apiserver changes see Version Skew Strategy .

Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.?

No.

Monitoring Requirements

How can an operator determine if the feature is in use by workloads?

Cluster administrator can verify audit entries looking for kubectl invocations targeting scale, status and resize subresources.

How can someone using this feature know that it is working for their instance?

N/A

What are the reasonable SLOs (Service Level Objectives) for the above SLIs?

Since this functionality doesn’t heavily modify kube-apiserver I’d expected the SLO defined here to apply.

What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service?

N/A

Are there any missing metrics that would be useful to have to improve observability of this feature?

N/A

Dependencies

Does this feature depend on any specific services running in the cluster?

No.

Scalability

Will enabling / using this feature result in any new API calls?

  • API call type (e.g. PATCH pods): GET, PATCH, PUT /<subresource>

  • Estimated throughput: Negligible, because it’s human initiated. At maximum, each command would involve two calls: 1 read and 1 mutate.

  • Originating component(s): kubectl

Will enabling / using this feature result in introducing new API types?

No

Will enabling / using this feature result in any new calls to the cloud provider?

No

Will enabling / using this feature result in increasing size or count of the existing API objects?

No

Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?

No

Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, …) in any components?

No

Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)?

No

Troubleshooting

How does this feature react if the API server and/or etcd is unavailable?

kubectl is not resilient to API server unavailability.

What are other known failure modes?

N/A

What steps should be taken if SLOs are not being met to determine the problem?

N/A

Implementation History

2021-03-01: Initial POC PR created 2021-04-06: KEP proposed 2021-04-07: Demo in SIG CLI meeting 2022-05-25: PR for alpha implementation merged 2023-01-12: KEP graduated to Beta 2023-03-15: e2e test added for KEP as part of beta graduation 2025-01-23: KEP graduated to Stable

Alternatives

Alternatives would be to use curl commands directly to update subresources.