Each directory with tests should have a sig label in its OWNERS file to
document which SIG is responsible for the test. Otherwise SIG Testing gets
pinged by SIG Release as the default owner of all tests when a test fails.
Some integration test packages are either empty (objectmeta) or useless with
unclear owner (configmap); those get removed.
When SIG Release triaged testgrid failures for unit or integration
test jobs, they often ended up reaching out to SIG Testing as the
owner of those jobs.
Injecting the same [sig-...] prefix as in E2E tests makes it more
obvious which SIG needs to handle specific test failures. This
avoids the manual step of looking up source code and finding
the relevant OWNERS file: those steps are now done automatically
by prune-junit-xml.
During unit or integration testing we collapsed all tests in a "class" (=
package) into a single entry, with a shorter name. But the suite name was left
unchanged, with the result that testgrid showed
e.g. "k8s.io/kubernetes/test/integration/scheduler/preemption.preemption".
Shortening the suite name in the same way as the class name turns that into
"k8s.io/kubernetes/test/integration/scheduler.preemption".
Spyglass uses the class name and shows
"k8s.io/kubernetes/test/integration/scheduler: preemption", with and without
this change.
When stripping out log messages from the failure text, the original text gets
stored as <system-out>. That part then got lost when reducing tests. Instead of
dropping it, it needs to be joined from all failed tests. Same for
<system-err>, although that isn't used yet.
Repeating the same "Failed" message text doesn't add any
information. Separating with a blank line is more readable.
Before:
<failure message="Failed; Failed; Failed" type="">
...
--- FAIL: TestFrontProxyConfig/WithoutUID (64.89s) ; === RUN TestFrontProxyConfig/WithUID
After:
<failure message="Failed" type="">
...
--- FAIL: TestFrontProxyConfig/WithoutUID (64.89s)
=== RUN TestFrontProxyConfig/WithUID
Appending a semicolon after some text is unnecessary if it's the last
entry. This led to visually distracting extra semicolons in Spyglass which
looked like a bug in Spyglass.
Now the code checks if a semicolon is necessary before inserting it.
In Go unit tests, the entire test output becomes the failure message because
`go test` doesn't track why a test fails. This can make the failure message
pretty large, in particular in integration tests.
We cannot identify the real failure either because Kubernetes has no convention
for how to format test failures. What we can do is recognize log output added
by klog.
prune-junit-xml now moves the full text to to the test output and only keep
those lines in the failure which are not from klog.
The klog output parsing might eventually get moved to
k8s.io/logtools/logparse. For now it is developed as a sub-package of
prune-junit-xml.
Over time the size of our junit xml has exploded to the point where
test-grid fails to process them. We still have the original/full
*.stdout files from where the junit xml files are generated from so the
junit xml files need NOT have the fill/exact output for
processing/display. So let us prune the large messages with an
indicator that we have "[... clipped...]" some of the content so folks
can see that they have to consult the full *.stdout files.
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
