kubernetes/test/e2e
Omer Tuchfeld eb317ecd40 Fix capture loop vars in parallel or ginkgo tests
Fixes instances of #98213 (to ultimately complete #98213 linting is
required).

This commit fixes a few instances of a common mistake done when writing
parallel subtests or Ginkgo tests (basically any test in which the test
closure is dynamically created in a loop and the loop doesn't wait for
the test closure to complete).

I'm developing a very specific linter that detects this king of mistake
and these are the only violations of it it found in this repo (it's not
airtight so there may be more).

In the case of Ginkgo tests, without this fix, only the last entry in
the loop iteratee is actually tested. In the case of Parallel tests I
think it's the same problem but maybe a bit different, iiuc it depends
on the execution speed.

Waiting for the CI to confirm the tests are still passing, even after
this fix - since it's likely it's the first time those test cases are
executed - they may be buggy or testing code that is buggy.

Another instance of this is in `test/e2e/storage/csi_mock_volume.go` and
is still failing so it has been left out of this commit and will be
addressed in a separate one
2022-08-15 16:28:50 +02:00
..
apimachinery Remove feature 2022-08-09 19:23:27 +00:00
apps Add simple e2e verifying TimeZone support for CronJob 2022-08-02 16:53:10 +02:00
architecture update ginkgo from v1 to v2 and gomega to 1.19.0 2022-07-08 10:44:46 +08:00
auth update ginkgo from v1 to v2 and gomega to 1.19.0 2022-07-08 10:44:46 +08:00
autoscaling Merge pull request #111346 from piotrnosek/hpa-tests-2 2022-08-09 03:14:50 -07:00
chaosmonkey ginkgo.By can only be used inside a runnable node 2022-07-08 10:46:11 +08:00
cloud Fix capture loop vars in parallel or ginkgo tests 2022-08-15 16:28:50 +02:00
common Merge pull request #111440 from verb/111025-ec-conformance 2022-08-08 15:35:57 -07:00
framework Merge pull request #111346 from piotrnosek/hpa-tests-2 2022-08-09 03:14:50 -07:00
instrumentation Promote Event e2e test to Conformance 2022-07-25 09:15:36 +12:00
kubectl Adjust testing for server-side validation as default 2022-07-18 16:16:00 +00:00
lifecycle update ginkgo from v1 to v2 and gomega to 1.19.0 2022-07-08 10:44:46 +08:00
network Fix e2e network dns_configmap test 2022-08-04 21:04:02 +00:00
node Fix capture loop vars in parallel or ginkgo tests 2022-08-15 16:28:50 +02:00
perftype hack/update-bazel.sh 2021-02-28 15:17:29 -08:00
reporters Migrate ProgressReporter to Ginkgo V2 2022-07-08 10:46:11 +08:00
scheduling Make scheduling e2e tests run PSa-restricted pods 2022-07-26 14:32:51 +02:00
storage Fix capture loop vars in parallel or ginkgo tests 2022-08-15 16:28:50 +02:00
testing-manifests Switching everything to use pause:3.8 2022-07-21 14:53:15 -07:00
upgrades update ginkgo from v1 to v2 and gomega to 1.19.0 2022-07-08 10:44:46 +08:00
windows Merge pull request #111609 from marosset/hpc-test-updates 2022-08-01 15:05:16 -07:00
e2e-example-config.json
e2e.go Custom reporter of Junit report is no longer needed 2022-07-08 10:46:11 +08:00
e2e_test.go Revert "e2e: Trim junit reporter to adapt with testgrid" 2022-08-03 15:45:09 +02:00
README.md test/e2e: Add ownership info to README 2021-03-17 16:48:11 -04:00
suites.go io/ioutil has already been deprecated in golang 1.16, so replace all ioutil with io and os 2022-02-03 05:32:12 +08:00

test/e2e

This is home to e2e tests used for presubmit, periodic, and postsubmit jobs.

Some of these jobs are merge-blocking, some are release-blocking.

e2e test ownership

All e2e tests must adhere to the following policies:

  • the test must be owned by one and only one SIG
  • the test must live in/underneath a sig-owned package matching pattern: test/e2e/[{subpath}/]{sig}/..., e.g.
    • test/e2e/auth - all tests owned by sig-auth
    • test/e2e/common/storage - all tests common to cluster-level and node-level e2e tests, owned by sig-node
    • test/e2e/upgrade/apps - all tests used in upgrade testing, owned by sig-apps
  • each sig-owned package should have an OWNERS file defining relevant approvers and labels for the owning sig, e.g.
# test/e2e/node/OWNERS
# See the OWNERS docs at https://go.k8s.io/owners

approvers:
- alice
- bob
- cynthia
emeritus_approvers:
- dave
reviewers:
- sig-node-reviewers
labels:
- sig/node
  • packages that use {subpath} should have an imports.go file importing sig-owned packages (for ginkgo's benefit), e.g.
// test/e2e/common/imports.go
package common

import (
	// ensure these packages are scanned by ginkgo for e2e tests
	_ "k8s.io/kubernetes/test/e2e/common/network"
	_ "k8s.io/kubernetes/test/e2e/common/node"
	_ "k8s.io/kubernetes/test/e2e/common/storage"
)
  • test ownership must be declared via a top-level SIGDescribe call defined in the sig-owned package, e.g.
// test/e2e/lifecycle/framework.go
package lifecycle

import "github.com/onsi/ginkgo"

// SIGDescribe annotates the test with the SIG label.
func SIGDescribe(text string, body func()) bool {
	return ginkgo.Describe("[sig-cluster-lifecycle] "+text, body)
}
// test/e2e/lifecycle/bootstrap/bootstrap_signer.go

package bootstrap

import (
	"github.com/onsi/ginkgo"
	"k8s.io/kubernetes/test/e2e/lifecycle"
)
var _ = lifecycle.SIGDescribe("[Feature:BootstrapTokens]", func() {
  /* ... */
  ginkgo.It("should sign the new added bootstrap tokens", func() {
    /* ... */
  })
  /* etc */
})

These polices are enforced:

  • via the merge-blocking presubmit job pull-kubernetes-verify
  • which ends up running hack/verify-e2e-test-ownership.sh
  • which can also be run via make verify WHAT=e2e-test-ownership

more info

See kubernetes/community/.../e2e-tests.md