KEP-6356: publishing ktesting

Implementation History
STABLE Implementable
Created 2026-09-14
Latest v1.38
Milestones
Stable v1.38
Ownership
Owning SIG
SIG Testing
Primary Authors
@pohly

KEP-6356: publishing ktesting

Release Signoff Checklist

Items marked with (R) are required prior to targeting to a milestone / release.

  • (R) Enhancement issue in release milestone, which links to KEP dir in kubernetes/enhancements (not the initial KEP PR)
  • (R) KEP approvers have approved the KEP status as implementable
  • (R) Design details are appropriately documented
  • (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
    • e2e Tests for all Beta API Operations (endpoints) (not applicable and hence done)
    • (R) Ensure GA e2e tests meet requirements for Conformance Tests (not applicable and hence done)
    • (R) Minimum Two Week Window for GA e2e tests to prove flake free (not applicable and hence done)
  • (R) Graduation criteria is in place
  • (R) Production readiness review completed
  • (R) Production readiness review approved
  • “Implementation History” section is up-to-date for milestone
  • User-facing documentation has been created in kubernetes/website , for publication to kubernetes.io (not applicable, docs are automatically provided via go docs and https://pkg.go.dev , hence done)
  • Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes

Summary

ktesting is a set of Go packages that support writing unit, integration and E2E tests. Two of them are currently located in k8s.io/kubernetes/test/utils:

  • k8s.io/kubernetes/test/utils/ktesting: abstraction around Go testing and Ginkgo
  • k8s.io/kubernetes/test/utils/client-go/ktesting: tailored towards testing Kubernetes as a client with client-go calls

The proposal in this KEP is to publish k8s.io/kubernetes/test/utils/ktesting in a new k8s.io/ktesting staging repo and k8s.io/kubernetes/test/utils/client-go/ktesting as a k8s.io/client-go/ktesting sub-package.

Motivation

The main motivation is to enable the use of ktesting in all code developed in the kubernetes/kubernetes repository, including staging repos. Currently it can be used only in core Kubernetes tests, not staging repos, because that would cause an import cycle.

A secondary motivation is to enable use outside of Kubernetes. Maybe being able to use it will motivate other developers to contribute to it.

Goals

  • publish the core ktesting package as k8s.io/ktesting
  • move the client-go support to k8s.io/client-go/ktesting

Non-Goals

  • define a policy when to use or not to use ktesting
  • large-scale conversion of tests which aren’t using ktesting yet; this can be decided by code owners on a case-by-case basis

Proposal

First the new k8s.io/ktesting staging repo needs to be created. Then a single PR will move the existing code and adapt package names to keep everything compiling without errors.

No further work is needed. The ktesting packages are fully documented, so go doc and https://pkg.go.dev will provide the necessary documentation for developers.

The publicly available ktesting packages will have the same Go API stability goals as other staging repos:

  • no plan to stabilize a v1 package API
  • API breaks will be avoided as much as possible, but may happen if needed

User Stories

Using ktesting in k8s.io/dynamic-resource-allocation

ktesting was developed alongside DRA and is used in various in-tree tests. It is jarring that the same functionality is not available when working on the DRA code in staging, k8s.io/dynamic-resource-allocation, because tests have to be written differently and behave differently.

Once ktesting is available for use in staging repos, tests in k8s.io/dynamic-resource-allocation can be updated to be consistent.

Moving pkg/scheduler to staging

Some code under pkg/scheduler uses test/utils/ktesting. To move that code into k8s.io/kube-scheduler, either tests must be rewritten or ktesting must be made usable from k8s.io/kube-scheduler.

Risks and Mitigations

Outside usage of ktesting might lead to an expectation of Go API stability. ktesting is designed as much as possible to allow future extensions without breaking Go APIs (concrete type instead of interface, functional configuration parameters, etc.). Should the need arise to break an API, then the existing documentation makes it clear that this is allowed. As a staging repo, such a change can be made in a single atomic PR.

Outside usage of ktesting might cause feature requests or bug reports. Bug reports are useful because they help improving the code. Feature requests may provide useful ideas, but there is no obligation to implement new features. Pull requests which implement new features may get reviewed and merged, but as always only if reviewers have the time and motivation to do so.

Because ktesting is never used in production code, there is no risk that it’ll ever cause a security issue in components using it. A CI might be affected, but this is not different from running tests developed by a contributor.

Design Details

For further information about the design and features of ktesting, see test/utils/ktesting docs or better (once available) https://pkg.go.dev/k8s.io/kubernetes@v1.38.0-alpha.1/test/utils/ktesting

Test Plan

[X] I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement.

Prerequisite testing updates
Unit tests
  • k8s.io/kubernetes/test/utils/ktesting: 2026-09-14 - 89.6%
  • k8s.io/kubernetes/test/utils/ktesting/format: 2026-09-14 - 80.0%
  • k8s.io/kubernetes/test/utils/client-go/ktesting: 2026-09-14 - 80.9%
Integration tests

e2e framework

e2e tests

Not applicable because ktesting itself does not run in a cluster.

Graduation Criteria

GA

  • ktesting published as proposed

Deprecation

None.

Upgrade / Downgrade Strategy

Not applicable.

Version Skew Strategy

Not applicable.

Production Readiness Review Questionnaire

Feature Enablement and Rollback

Not applicable.

How can this feature be enabled / disabled in a live cluster?

Not applicable.

Does enabling the feature change any default behavior?

Not applicable.

Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?

Not applicable.

What happens if we reenable the feature if it was previously rolled back?

Not applicable.

Are there any tests for feature enablement/disablement?

Not applicable.

Rollout, Upgrade and Rollback Planning

Not applicable.

How can a rollout or rollback fail? Can it impact already running workloads?

Not applicable.

What specific metrics should inform a rollback?

Not applicable.

Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?

Not applicable.

Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.?

Not applicable.

Monitoring Requirements

Not applicable.

How can an operator determine if the feature is in use by workloads?

Not applicable.

How can someone using this feature know that it is working for their instance?

Not applicable.

What are the reasonable SLOs (Service Level Objectives) for the enhancement?

Not applicable.

What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service?

Not applicable.

Are there any missing metrics that would be useful to have to improve observability of this feature?

Not applicable.

Dependencies

The core ktesting has a tightly controlled set of dependencies besides the standard library:

  • github.com/go-logr
  • github.com/onsi/gomega
  • go.uber.org/goleak (test-only)
  • k8s.io/klog/v2/ktesting
  • sigs.k8s.io/yaml

This should make it usable in a wide variety of other modules. If some of these dependencies are not acceptable for a module, then ktesting cannot be used there.

client-go currently depends on all of these already, except for Gomega. Instead it depends on testify, which has worse indirect dependencies than Gomega. In particular, testify depends on go-spew, an unwanted dependency in Kubernetes.

It’s under debate whether test-only dependencies should be considered (see “analyze production dependencies” ). If we care, then replacing testify with Gomega might be a good next step. See “Assertions” for an explanation how and why ktesting integrates support for Gomega.

Does this feature depend on any specific services running in the cluster?

No.

Scalability

Not applicable because not used in production code.

Will enabling / using this feature result in any new API calls?

No.

Will enabling / using this feature result in introducing new API types?

No.

Will enabling / using this feature result in any new calls to the cloud provider?

No.

Will enabling / using this feature result in increasing size or count of the existing API objects?

No.

Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?

No.

Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, …) in any components?

No.

Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)?

No.

Troubleshooting

Not applicable.

How does this feature react if the API server and/or etcd is unavailable?

Not applicable.

What are other known failure modes?

None.

What steps should be taken if SLOs are not being met to determine the problem?

Not applicable.

Implementation History

ktesting was born out of the frustration of having to extend all functions along a deep call tree such they pass through a generic client in addition to several other client-go related parameters and the context. The actual code then created arbitrary objects from YAML files, with automatic cleanup and reporting errors as test failures. This would have been useful for both Go and Ginkgo tests, except that the lack of a common API prevented the code reuse.

Nowadays ktesting provides that common API and access to everything tests need (context, access to client-go clients) with a single parameter. It has already been cleaned up to make it directly suitable for publishing.

Development of ktesting was covered in:

Drawbacks

None. Development of ktesting will continue as before.

Alternatives

Publishing as separate repo, usage in Kubernetes

This would be viable. However, it is less desirable than a staging repo because future changes will be harder to make: first the out-of-tree repo needs to be updated, then the update needs to be vendored, and only then can tests inside Kubernetes use a new feature.

Setting up test jobs which ensure that updates don’t break Kubernetes will cause further work and will be slightly more costly than running more tests in existing jobs because of the cold Go caches.

A separate repo would have the advantage that it would be possible to version releases separately, possibly even with a v1. However, such a v1 and this kind of manual release control are not goals. If they become one, then we can still switch to modifying and releasing directly in the github.com/kubernetes/ktesting repo, with no visible difference for consumers (same import path).

Publishing as separate repo, no usage in Kubernetes

Besides having to rewrite existing tests before this is even feasible, it would also lead to the same undesirable situation as with sigs.k8s.io/e2e-framework: SIG Testing would directly or indirectly own code that doesn’t benefit the development of Kubernetes itself.

Infrastructure Needed

A staging repo.